Re: [PATCH v1 0/2] Btrfs-progs: commands resolve inode and resolve logical
On 08.07.2011 01:19, Goffredo Baroncelli wrote: On 07/07/2011 06:01 PM, Jan Schmidt wrote: The kernel patch series just sent (Subject: Btrfs: scrub: print path to corrupted files and trigger nodatasum fixup) introduces two new ioctls to do in-kernel filesystem path construction. This series provides the corresponding userspace changes, adding two new commands to the btrfs utility: Which is the aim of these commands ? It seems more a debug utilities than a standard command. If so, these commands may be put under a new group called debug or test or whichever we decided to use. But, please, highlight the fact that these commands aren't for a general use. Well, in my opinion, they are. Finding files for an inode is not that exotic, is it? The command to find files for logical addresses is useful until each and every error message in the kernel log does that resolving itself. Currently, we log logical addresses everywhere. Once that's changed, this one becomes more like a debug command, I agree. I suggest to use btrfs debug resolve ... Or better btrfs inspect resolve ... I don't give much in command names (I admit they should make sense), I'm happy to insert an inspect there. Although btrfs inspect resolve inode sounds strange to me. Any other opinions? I suggest we'd better do the naming process in #btrfs. -- btrfs resolve inode [-v] inode path resolves an inode to all filesystem paths local to the fs mounted at path. -v print count of returned and missed paths btrfs resolve logical [-v] [-P] logical path resolves a logical address to all filesystem paths in the file system mounted at path and all its subvolumes. -v print count of returned and missed inode/offset/root triples -P do not resolve the path but stop after finding all inodes at this logical address and print them instead -- These patches are based on Hugo's current integration branch. Please try them out and report bugs here. I'll send an update to the manpages later. Please update the man pages at the same time of the code. Develop the man page coupled with the code may help to design a good interface (from an user point of view) and to explain better the aim of the new command. Agreed. Next version will come with man pages. -Jan -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
How to get the default subvolume?
Hi, I know I can set the default subvolume for a btrfs fs using sudo btrfs subvolume set-default 256 /btrfs/mnt But after that, how can get the default subvolume name? In my opinion, btrfs-progs should provide btrfs subvolume get-default /btrfs/mnm to get the default subvolume id and name, I think it is very easy to do this in btrfs-progs and btrfs kernel space. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: How to get the default subvolume?
On Fri, Jul 08, 2011 at 05:58:02PM +0800, Yi Yang wrote: I know I can set the default subvolume for a btrfs fs using sudo btrfs subvolume set-default 256 /btrfs/mnt But after that, how can get the default subvolume name? In my opinion, btrfs-progs should provide btrfs subvolume get-default /btrfs/mnm to get the default subvolume id and name, I think it is very easy to do this in btrfs-progs and btrfs kernel space. I don't think the current btrfs-progs will do it. As you pointed out though, it's pretty easy to write the code to get the information (I implemented it for btrfs-gui without any additional kernel code, for example). We just need someone to implement it. :) I'd do it myself, but there's about a dozen things higher up my priority list right now. Hugo. -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk === PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- Welcome to Rivendell, Mr Anderson... --- signature.asc Description: Digital signature
Re: How to get the default subvolume?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 08.07.2011 11:58, Hugo Mills wrote: On Fri, Jul 08, 2011 at 05:58:02PM +0800, Yi Yang wrote: I know I can set the default subvolume for a btrfs fs using sudo btrfs subvolume set-default 256 /btrfs/mnt But after that, how can get the default subvolume name? In my opinion, btrfs-progs should provide btrfs subvolume get-default /btrfs/mnm to get the default subvolume id and name, I think it is very easy to do this in btrfs-progs and btrfs kernel space. I don't think the current btrfs-progs will do it. As you pointed out though, it's pretty easy to write the code to get the information (I implemented it for btrfs-gui without any additional kernel code, for example). We just need someone to implement it. :) I'd do it myself, but there's about a dozen things higher up my priority list right now. Well, I'd be happy to try it. If it's fine, I will try to follow the implementation from btrfs-gui. Andreas -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQIcBAEBAgAGBQJOFtWBAAoJEJIcBJ3+Xkgi/loP/jBB2BnrDvdRrN+gxP5XcCr8 62ZgP2XxO/MrIH5Xc4iNfUlMF8Fnttjr8LvjpbP3Q2Vfb2ogPPf6ALpBxtCOhaHJ yAFE4CvGon7f2sTGZO2EJxf4eSSoFUd19je+knPZOdyIDhc1lhXVp4LBeoPKySRe GDBQtv1uqYx79hD24R5vU5uK1xMDTVE5HHCy8jZtoaVGU4YO87u7vFfNJKqAmS/W tZbT3wkrod/HHo8sxS4ZKzPx1c0Ph/F3g/P8SWSHW5dmK/dSadRVe+Io09tyd6uu GfGMMtEs6lVjHq/gn4ebDZwk3pFmCACnCxk6cE5imGWOIvftLMCFRZkFS41BVn7s pAz+JEp7NaICKo3rbFlEZdvteBB0AaJTCeo2wAP0xD6aBEdT5MhxHZQCGi6PeRDn 2GTqw2gT+urTVa7Rr6O2PIQlC0mjFit6dVYcFzPIP/X+cXozDnuNeUmPD10VlEjN PHIOJjNX3Zod8KknQc/NX0kn3TcHZptX0EWJhtnC0X4kHMxZOUrfnlIMTjCP23pX TA4Zjc68fe5mDm75PnbT6h4QoLa5h3j8TWf8fYYFBC/FOcb4NFm4iJEA6AG7CS1l MExGgd619NWUXHDpjjhTTzerBHF+1D5XPNc0lnb7WNkjzfBQD5JfkFzCp3lzTylU dpaD5umGdM+1DlrD8NP/ =vM5/ -END PGP SIGNATURE- -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: How to get the default subvolume?
That's great, can you share your source code with me? I'm very eager to get this now :-) -Original Message- From: Hugo Mills [mailto:h...@carfax.org.uk] Sent: Friday, July 08, 2011 5:58 PM To: Yang, Yi Y Cc: linux-btrfs@vger.kernel.org Subject: Re: How to get the default subvolume? On Fri, Jul 08, 2011 at 05:58:02PM +0800, Yi Yang wrote: I know I can set the default subvolume for a btrfs fs using sudo btrfs subvolume set-default 256 /btrfs/mnt But after that, how can get the default subvolume name? In my opinion, btrfs-progs should provide btrfs subvolume get-default /btrfs/mnm to get the default subvolume id and name, I think it is very easy to do this in btrfs-progs and btrfs kernel space. I don't think the current btrfs-progs will do it. As you pointed out though, it's pretty easy to write the code to get the information (I implemented it for btrfs-gui without any additional kernel code, for example). We just need someone to implement it. :) I'd do it myself, but there's about a dozen things higher up my priority list right now. Hugo. -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk === PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- Welcome to Rivendell, Mr Anderson... --- -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: How to get the default subvolume?
On Fri, Jul 08, 2011 at 06:06:02PM +0800, Yang, Yi Y wrote: That's great, can you share your source code with me? I'm very eager to get this now :-) http://carfax.org.uk/btrfs-gui http://git.darksatanic.net/repo/btrfs-gui.git/ You probably want the btrfsgui.hlp.subvol.sv_list() function. Hugo. -Original Message- From: Hugo Mills [mailto:h...@carfax.org.uk] Sent: Friday, July 08, 2011 5:58 PM To: Yang, Yi Y Cc: linux-btrfs@vger.kernel.org Subject: Re: How to get the default subvolume? On Fri, Jul 08, 2011 at 05:58:02PM +0800, Yi Yang wrote: I know I can set the default subvolume for a btrfs fs using sudo btrfs subvolume set-default 256 /btrfs/mnt But after that, how can get the default subvolume name? In my opinion, btrfs-progs should provide btrfs subvolume get-default /btrfs/mnm to get the default subvolume id and name, I think it is very easy to do this in btrfs-progs and btrfs kernel space. I don't think the current btrfs-progs will do it. As you pointed out though, it's pretty easy to write the code to get the information (I implemented it for btrfs-gui without any additional kernel code, for example). We just need someone to implement it. :) I'd do it myself, but there's about a dozen things higher up my priority list right now. Hugo. -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk === PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- vi: The core of evil. --- signature.asc Description: Digital signature
RE: How to get the default subvolume?
From your source, you defined some structures using python and use them to get and parse btrfs metadata, very advanced and complicated :-), do you know there is any library to provide similar APIs in order that I can use some advanced APIs and don't need to worry about frequent changes of btrfs superblock and other metadata. -Original Message- From: linux-btrfs-ow...@vger.kernel.org [mailto:linux-btrfs-ow...@vger.kernel.org] On Behalf Of Hugo Mills Sent: Friday, July 08, 2011 6:13 PM To: Yang, Yi Y Cc: linux-btrfs@vger.kernel.org Subject: Re: How to get the default subvolume? On Fri, Jul 08, 2011 at 06:06:02PM +0800, Yang, Yi Y wrote: That's great, can you share your source code with me? I'm very eager to get this now :-) http://carfax.org.uk/btrfs-gui http://git.darksatanic.net/repo/btrfs-gui.git/ You probably want the btrfsgui.hlp.subvol.sv_list() function. Hugo. -Original Message- From: Hugo Mills [mailto:h...@carfax.org.uk] Sent: Friday, July 08, 2011 5:58 PM To: Yang, Yi Y Cc: linux-btrfs@vger.kernel.org Subject: Re: How to get the default subvolume? On Fri, Jul 08, 2011 at 05:58:02PM +0800, Yi Yang wrote: I know I can set the default subvolume for a btrfs fs using sudo btrfs subvolume set-default 256 /btrfs/mnt But after that, how can get the default subvolume name? In my opinion, btrfs-progs should provide btrfs subvolume get-default /btrfs/mnm to get the default subvolume id and name, I think it is very easy to do this in btrfs-progs and btrfs kernel space. I don't think the current btrfs-progs will do it. As you pointed out though, it's pretty easy to write the code to get the information (I implemented it for btrfs-gui without any additional kernel code, for example). We just need someone to implement it. :) I'd do it myself, but there's about a dozen things higher up my priority list right now. Hugo. -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk === PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- vi: The core of evil. --- -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
btrfs hang in flush-btrfs-5
Hi - I'm trying btrfs with kernel 2.6.38.8-32.fc15.x86_64 (a Fedora kernel). I'm just doing a tar-to-tar copy onto the file system with compress- force=zlib. Here are some traces of the stuck processes. flush-btrfs-5 seems to be stuck: Jul 8 11:49:40 xback2 kernel: [74920.681032] flush-btrfs-5 D 88003c7bae60 0 11712 2 0x0080 Jul 8 11:49:40 xback2 kernel: [74920.681032] 88001842f750 0046 ce5a 88003c7bae60 Jul 8 11:49:40 xback2 kernel: [74920.681032] 88001842ffd8 88001842ffd8 00013840 00013840 Jul 8 11:49:40 xback2 kernel: [74920.681032] 88005b819730 88003c7bae60 88005fd140c8 88005feb2188 Jul 8 11:49:40 xback2 kernel: [74920.681032] Call Trace: Jul 8 11:49:40 xback2 kernel: [74920.681032] [810d80c7] ? sync_page+0x0/0x4f Jul 8 11:49:40 xback2 kernel: [74920.681032] [8147439c] io_schedule+0x47/0x62 Jul 8 11:49:40 xback2 kernel: [74920.681032] [810d8112] sync_page+0x4b/0x4f Jul 8 11:49:40 xback2 kernel: [74920.681032] [8147482f] __wait_on_bit_lock+0x46/0x8f Jul 8 11:49:40 xback2 kernel: [74920.681032] [810d8075] __lock_page+0x66/0x68 Jul 8 11:49:40 xback2 kernel: [74920.681032] [8106f2ab] ? wake_bit_function+0x0/0x31 Jul 8 11:49:40 xback2 kernel: [74920.681032] [a0430cf9] lock_page+0x3a/0x3e [btrfs] Jul 8 11:49:40 xback2 kernel: [74920.681032] [a04310a6] lock_delalloc_pages+0xad/0x1af [btrfs] Jul 8 11:49:40 xback2 kernel: [74920.681032] [a0432f7c] find_lock_delalloc_range.constprop.9+0xc8/0x1ba [btrfs] Jul 8 11:49:40 xback2 kernel: [74920.681032] [a0433866] __extent_writepage+0x15c/0x582 [btrfs] Jul 8 11:49:40 xback2 kernel: [74920.681032] [8122d488] ? radix_tree_gang_lookup_tag_slot+0x81/0xa2 Jul 8 11:49:40 xback2 kernel: [74920.681032] [81474867] ? __wait_on_bit_lock+0x7e/0x8f Jul 8 11:49:40 xback2 kernel: [74920.681032] [a0433dd0] extent_write_cache_pages.constprop.6+0x144/0x28f [btrfs] Jul 8 11:49:40 xback2 kernel: [74920.681032] [81475f0e] ? common_interrupt+0xe/0x13 Jul 8 11:49:40 xback2 kernel: [74920.681032] [a043417b] extent_writepages+0x3f/0x50 [btrfs] Jul 8 11:49:40 xback2 kernel: [74920.681032] [8113e27a] ? list_move+0x29/0x30 Jul 8 11:49:40 xback2 kernel: [74920.681032] [a041af1d] ? btrfs_get_extent+0x0/0x74f [btrfs] Jul 8 11:49:40 xback2 kernel: [74920.681032] [a041ade3] btrfs_writepages+0x28/0x2a [btrfs] Jul 8 11:49:40 xback2 kernel: [74920.681032] [810e05d5] do_writepages+0x21/0x2a Jul 8 11:49:40 xback2 kernel: [74920.681032] [8113e386] writeback_single_inode+0x96/0x194 Jul 8 11:49:40 xback2 kernel: [74920.681032] [8113e6db] writeback_sb_inodes+0xa1/0x12b Jul 8 11:49:40 xback2 kernel: [74920.681032] [8113f4f0] writeback_inodes_wb+0x163/0x175 Jul 8 11:49:40 xback2 kernel: [74920.681032] [8113f741] wb_writeback+0x23f/0x35a Jul 8 11:49:40 xback2 kernel: [74920.681032] [81080b7b] ? arch_local_irq_save+0x15/0x1b Jul 8 11:49:40 xback2 kernel: [74920.681032] [8113f8e2] wb_do_writeback+0x86/0x19d Jul 8 11:49:40 xback2 kernel: [74920.681032] [81060bb4] ? process_timeout+0x0/0x10 Jul 8 11:49:40 xback2 kernel: [74920.681032] [8113fa81] bdi_writeback_thread+0x88/0x1e5 Jul 8 11:49:40 xback2 kernel: [74920.681032] [8113f9f9] ? bdi_writeback_thread+0x0/0x1e5 Jul 8 11:49:40 xback2 kernel: [74920.681032] [8106ebaf] kthread+0x84/0x8c Jul 8 11:49:40 xback2 kernel: [74920.681032] [8100a9e4] kernel_thread_helper+0x4/0x10 Jul 8 11:49:40 xback2 kernel: [74920.681032] [8106eb2b] ? kthread+0x0/0x8c Jul 8 11:49:40 xback2 kernel: [74920.681032] [8100a9e0] ? kernel_thread_helper+0x0/0x10 Here is the state of the tar which is stuck in D: Jul 8 11:49:40 xback2 kernel: [74920.681032] tar D 88005fc0f440 0 13171 11702 0x0084 Jul 8 11:49:40 xback2 kernel: [74920.681032] 88003aa5b468 0086 0001 880059bdc590 Jul 8 11:49:40 xback2 kernel: [74920.681032] 88003aa5bfd8 88003aa5bfd8 00013840 00013840 Jul 8 11:49:40 xback2 kernel: [74920.681032] 88005ba1c590 880059bdc590 88005fc140c8 00015feb58a8 Jul 8 11:49:40 xback2 kernel: [74920.681032] Call Trace: Jul 8 11:49:40 xback2 kernel: [74920.681032] [810d80c7] ? sync_page+0x0/0x4f Jul 8 11:49:40 xback2 kernel: [74920.681032] [8147439c] io_schedule+0x47/0x62 Jul 8 11:49:40 xback2 kernel: [74920.681032] [810d8112] sync_page+0x4b/0x4f Jul 8 11:49:40 xback2 kernel: [74920.681032] [8147482f] __wait_on_bit_lock+0x46/0x8f Jul 8 11:49:40 xback2 kernel: [74920.681032] [810d8075] __lock_page+0x66/0x68 Jul 8 11:49:40 xback2 kernel: [74920.681032] [8106f2ab] ?
Re: How to get the default subvolume?
On Fri, Jul 08, 2011 at 06:28:54PM +0800, Yang, Yi Y wrote: From your source, you defined some structures using python and use them to get and parse btrfs metadata, very advanced and complicated :-), do you know there is any library to provide similar APIs in order that I can use some advanced APIs and don't need to worry about frequent changes of btrfs superblock and other metadata. The fundamental btrfs data structures (which you'll find defined in ctree.h in the btrfs-progs source, and which are mirrored in my python code) won't change much, since they define the on-disk layout. Changing those structures creates incompatible filesystems, which is a Bad Thing, so it doesn't happen all that often. Hugo. PS. On most development mailing lists, it's conventional to write your replies below the text you're replying to, not above. -Original Message- From: linux-btrfs-ow...@vger.kernel.org [mailto:linux-btrfs-ow...@vger.kernel.org] On Behalf Of Hugo Mills Sent: Friday, July 08, 2011 6:13 PM To: Yang, Yi Y Cc: linux-btrfs@vger.kernel.org Subject: Re: How to get the default subvolume? On Fri, Jul 08, 2011 at 06:06:02PM +0800, Yang, Yi Y wrote: That's great, can you share your source code with me? I'm very eager to get this now :-) http://carfax.org.uk/btrfs-gui http://git.darksatanic.net/repo/btrfs-gui.git/ You probably want the btrfsgui.hlp.subvol.sv_list() function. Hugo. -Original Message- From: Hugo Mills [mailto:h...@carfax.org.uk] Sent: Friday, July 08, 2011 5:58 PM To: Yang, Yi Y Cc: linux-btrfs@vger.kernel.org Subject: Re: How to get the default subvolume? On Fri, Jul 08, 2011 at 05:58:02PM +0800, Yi Yang wrote: I know I can set the default subvolume for a btrfs fs using sudo btrfs subvolume set-default 256 /btrfs/mnt But after that, how can get the default subvolume name? In my opinion, btrfs-progs should provide btrfs subvolume get-default /btrfs/mnm to get the default subvolume id and name, I think it is very easy to do this in btrfs-progs and btrfs kernel space. I don't think the current btrfs-progs will do it. As you pointed out though, it's pretty easy to write the code to get the information (I implemented it for btrfs-gui without any additional kernel code, for example). We just need someone to implement it. :) I'd do it myself, but there's about a dozen things higher up my priority list right now. Hugo. -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk === PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- vi: The core of evil. --- signature.asc Description: Digital signature
Re: Memory leak?
Excerpts from Stephane Chazelas's message of 2011-07-08 08:44:29 -0400: 2011-07-06 09:11:11 +0100, Stephane Chazelas: 2011-07-03 13:38:57 -0600, cwillu: On Sun, Jul 3, 2011 at 1:09 PM, Stephane Chazelas stephane_chaze...@yahoo.fr wrote: [...] Now, on a few occasions (actually, most of the time), when I rsynced the data (about 2.5TB) onto the external drive, the system would crash after some time with Out of memory and no killable process. Basically, something in kernel was allocating the whole memory, then oom mass killed everybody and crash. [...] Look at the output of slabtop (should be installed by default, procfs package), before rsync for comparison, and during. Hi, so, no crash this time [...] Another attempt, again onto an empty drive, this time with 3.0.0-rc6. Exact same scenario. slabinfo: Got another invalid opcode in btrfs-fixup-0. That was in the middle of transfering a 3.6GB file. I've got one and only one of those during one boot time when doing that rsync. So the invalidate opcode in btrfs-fixup-0 is the big problem. We're either failing to write because we weren't able to allocate memory (and not dealing with it properly) or there is a bigger problem. Does the btrfs-fixup-0 oops come before or after the ooms? Please send along any oops output during the run. Only the first (earliest) oops matters. I would do two things. First, I'd turn off compress_force. There's no explicit reason for this, it just seems like the mostly likely place for a bug. If the oops comes after the oom messages, please grab a sysrq-t and sysrq-w output (you'll need some way to log the console if you do this). What we really need to see is what kswapd is doing. That's usually where the inodes get freed. -chris -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Memory leak?
2011-07-08 11:06:08 -0400, Chris Mason: [...] So the invalidate opcode in btrfs-fixup-0 is the big problem. We're either failing to write because we weren't able to allocate memory (and not dealing with it properly) or there is a bigger problem. Does the btrfs-fixup-0 oops come before or after the ooms? Hi Chris, thanks for looking into this. It comes long before. Hours before there's any problem. So it seems unrelated. Please send along any oops output during the run. Only the first (earliest) oops matters. There's always only one in between two reboots. I've sent two already, but here they are: Jul 1 18:25:57 [ cut here ] Jul 1 18:25:57 kernel BUG at /media/data/mattems/src/linux-2.6-3.0.0~rc5/debian/build/source_amd64_none/fs/btrfs/inode.c:1583! Jul 1 18:25:57 invalid opcode: [#1] SMP Jul 1 18:25:57 CPU 1 Jul 1 18:25:57 Modules linked in: sha256_generic cryptd aes_x86_64 aes_generic cbc dm_crypt fuse snd_pcm psmouse tpm_tis tpm i2c_i801 snd_timer snd soundcore snd_page_alloc i3200_edac tpm_bios serio_raw evdev pcspkr processor button thermal_sys i2c_core container edac_core sg sr_mod cdrom ext4 mbcache jbd2 crc16 dm_mod nbd btrfs zlib_deflate crc32c libcrc32c ums_cypress usb_storage sd_mod crc_t10dif uas uhci_hcd ahci libahci libata ehci_hcd e1000e scsi_mod usbcore [last unloaded: scsi_wait_scan] Jul 1 18:25:57 Jul 1 18:25:57 Pid: 747, comm: btrfs-fixup-0 Not tainted 3.0.0-rc5-amd64 #1 empty empty/Tyan Tank GT20 B5211 Jul 1 18:25:57 RIP: 0010:[a014b0f4] [a014b0f4] btrfs_writepage_fixup_worker+0xdb/0x120 [btrfs] Jul 1 18:25:57 RSP: 0018:8801483ffde0 EFLAGS: 00010246 Jul 1 18:25:57 RAX: RBX: ea000496a430 RCX: Jul 1 18:25:57 RDX: RSI: 06849000 RDI: 880071c1fcb8 Jul 1 18:25:57 RBP: 06849000 R08: 0008 R09: 8801483ffd98 Jul 1 18:25:57 R10: dead00200200 R11: dead00100100 R12: 880071c1fd90 Jul 1 18:25:57 R13: R14: 8801483ffdf8 R15: 06849fff Jul 1 18:25:57 FS: () GS:88014fd0() knlGS: Jul 1 18:25:57 CS: 0010 DS: ES: CR0: 8005003b Jul 1 18:25:57 CR2: f7596000 CR3: 00013def9000 CR4: 06e0 Jul 1 18:25:57 DR0: DR1: DR2: Jul 1 18:25:57 DR3: DR6: 0ff0 DR7: 0400 Jul 1 18:25:57 Process btrfs-fixup-0 (pid: 747, threadinfo 8801483fe000, task 88014672efa0) Jul 1 18:25:57 Stack: Jul 1 18:25:57 880071c1fc28 8800c70165c0 88011e61ca28 Jul 1 18:25:57 880146ef41c0 880146ef4210 880146ef41d8 Jul 1 18:25:57 880146ef41c8 880146ef4200 880146ef41e8 a01669fa Jul 1 18:25:57 Call Trace: Jul 1 18:25:57 [a01669fa] ? worker_loop+0x186/0x4a1 [btrfs] Jul 1 18:25:57 [813369ca] ? schedule+0x5ed/0x61a Jul 1 18:25:57 [a0166874] ? btrfs_queue_worker+0x24a/0x24a [btrfs] Jul 1 18:25:57 [a0166874] ? btrfs_queue_worker+0x24a/0x24a [btrfs] Jul 1 18:25:57 [8105faed] ? kthread+0x7a/0x82 Jul 1 18:25:57 [8133e524] ? kernel_thread_helper+0x4/0x10 Jul 1 18:25:57 [8105fa73] ? kthread_worker_fn+0x147/0x147 Jul 1 18:25:57 [8133e520] ? gs_change+0x13/0x13 Jul 1 18:25:57 Code: 41 b8 50 00 00 00 4c 89 f1 e8 d5 3b 01 00 48 89 df e8 fb ac f6 e0 ba 01 00 00 00 4c 89 ee 4c 89 e7 e8 ce 05 01 00 e9 4e ff ff ff 0f 0b eb fe 48 8b 3c 24 41 b8 50 00 00 00 4c 89 f1 4c 89 fa 48 Jul 1 18:25:57 RIP [a014b0f4] btrfs_writepage_fixup_worker+0xdb/0x120 [btrfs] Jul 1 18:25:57 RSP 8801483ffde0 Jul 1 18:25:57 ---[ end trace 9744d33381de3d04 ]--- Jul 4 12:50:51 [ cut here ] Jul 4 12:50:51 kernel BUG at /media/data/mattems/src/linux-2.6-3.0.0~rc5/debian/build/source_amd64_none/fs/btrfs/inode.c:1583! Jul 4 12:50:51 invalid opcode: [#1] SMP Jul 4 12:50:51 CPU 0 Jul 4 12:50:51 Modules linked in: lm85 dme1737 hwmon_vid coretemp ipmi_si ipmi_msghandler sha256_generic cryptd aes_x86_64 aes_generic cbc fuse dm_crypt snd_pcm snd_timer snd sg soundcore i3200_edac snd_page_alloc sr_mod processor tpm_tis i2c_i801 pl2303 pcspkr thermal_sys i2c_core tpm edac_core tpm_bios cdrom usbserial container evdev psmouse button serio_raw ext4 mbcache jbd2 crc16 dm_mod nbd btrfs zlib_deflate crc32c libcrc32c ums_cypress sd_mod crc_t10dif usb_storage uas uhci_hcd ahci libahci ehci_hcd libata e1000e usbcore scsi_mod [last unloaded: i2c_dev] Jul 4 12:50:51 Jul 4 12:50:51 Pid: 764, comm: btrfs-fixup-0 Not tainted 3.0.0-rc5-amd64 #1 empty empty/Tyan Tank GT20 B5211 Jul 4 12:50:51 RIP: 0010:[a00820f4] [a00820f4] btrfs_writepage_fixup_worker+0xdb/0x120 [btrfs] Jul 4 12:50:51 RSP:
Re: Memory leak?
2011-07-08 16:41:23 +0100, Stephane Chazelas: 2011-07-08 11:06:08 -0400, Chris Mason: [...] So the invalidate opcode in btrfs-fixup-0 is the big problem. We're either failing to write because we weren't able to allocate memory (and not dealing with it properly) or there is a bigger problem. Does the btrfs-fixup-0 oops come before or after the ooms? Hi Chris, thanks for looking into this. It comes long before. Hours before there's any problem. So it seems unrelated. Though every time I had the issue, there had been such an invalid opcode before. But also, I only had both the invalid opcode and memory issue when doing that rsync onto external hard drive. Please send along any oops output during the run. Only the first (earliest) oops matters. There's always only one in between two reboots. I've sent two already, but here they are: [...] I dug up the traces for before I switched to debian (thinking getting a newer kernel would improve matters) in case it helps: Jun 4 18:12:58 [ cut here ] Jun 4 18:12:58 kernel BUG at /build/buildd/linux-2.6.38/fs/btrfs/inode.c:1555! Jun 4 18:12:58 invalid opcode: [#2] SMP Jun 4 18:12:58 last sysfs file: /sys/devices/virtual/block/dm-2/dm/name Jun 4 18:12:58 CPU 0 Jun 4 18:12:58 Modules linked in: sha256_generic cryptd aes_x86_64 aes_generic dm_crypt psmouse serio_raw xgifb(C+) i3200_edac edac_core nbd btrfs zlib_deflate libcrc32c xenbus_probe_frontend ums_cypress usb_storage uas e1000e ahci libahci Jun 4 18:12:58 Jun 4 18:12:58 Pid: 416, comm: btrfs-fixup-0 Tainted: G D C 2.6.38-7-server #35-Ubuntu empty empty/Tyan Tank GT20 B5211 Jun 4 18:12:58 RIP: 0010:[a0099765] [a0099765] btrfs_writepage_fixup_worker+0x145/0x150 [btrfs] Jun 4 18:12:58 RSP: 0018:88003cfddde0 EFLAGS: 00010246 Jun 4 18:12:58 RAX: RBX: ea04ca88 RCX: Jun 4 18:12:58 RDX: 88003cfddd98 RSI: RDI: 8800152088b0 Jun 4 18:12:58 RBP: 88003cfdde30 R08: e8c09988 R09: 88003cfddd98 Jun 4 18:12:58 R10: R11: R12: 010ec000 Jun 4 18:12:58 R13: 880015208988 R14: R15: 010ecfff Jun 4 18:12:58 FS: () GS:88003fc0() knlGS: Jun 4 18:12:58 CS: 0010 DS: ES: CR0: 8005003b Jun 4 18:12:58 CR2: 00e73fe8 CR3: 30fcc000 CR4: 06f0 Jun 4 18:12:58 DR0: DR1: DR2: Jun 4 18:12:58 DR3: DR6: 0ff0 DR7: 0400 Jun 4 18:12:58 Process btrfs-fixup-0 (pid: 416, threadinfo 88003cfdc000, task 880036912dc0) Jun 4 18:12:58 Stack: Jun 4 18:12:58 880039c4e120 880015208820 88003cfdde90 880032da4b80 Jun 4 18:12:58 88003cfdde30 88003ce915a0 88003cfdde90 88003cfdde80 Jun 4 18:12:58 880036912dc0 88003ce915f0 88003cfddee0 a00c34f4 Jun 4 18:12:58 Call Trace: Jun 4 18:12:58 [a00c34f4] worker_loop+0xa4/0x3a0 [btrfs] Jun 4 18:12:58 [a00c3450] ? worker_loop+0x0/0x3a0 [btrfs] Jun 4 18:12:58 [81087116] kthread+0x96/0xa0 Jun 4 18:12:58 [8100cde4] kernel_thread_helper+0x4/0x10 Jun 4 18:12:58 [81087080] ? kthread+0x0/0xa0 Jun 4 18:12:58 [8100cde0] ? kernel_thread_helper+0x0/0x10 Jun 4 18:12:58 Code: 1f 80 00 00 00 00 48 8b 7d b8 48 8d 4d c8 41 b8 50 00 00 00 4c 89 fa 4c 89 e6 e8 37 d1 01 00 eb b6 48 89 df e8 8d 1a 07 e1 eb 9a 0f 0b 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 41 57 41 56 41 55 Jun 4 18:12:58 RIP [a0099765] btrfs_writepage_fixup_worker+0x145/0x150 [btrfs] Jun 4 18:12:58 RSP 88003cfddde0 Jun 4 18:12:58 ---[ end trace e5cf15794ff3ebdb ]--- And: Jun 5 00:58:10 BUG: Bad page state in process rsync pfn:1bfdf Jun 5 00:58:10 page:ea61f8c8 count:0 mapcount:0 mapping: (null) index:0x2300 Jun 5 00:58:10 page flags: 0x110(dirty) Jun 5 00:58:10 Pid: 1584, comm: rsync Tainted: G D C 2.6.38-7-server #35-Ubuntu Jun 5 00:58:10 Call Trace: Jun 5 00:58:10 [8111250b] ? dump_page+0x9b/0xd0 Jun 5 00:58:10 [8111260c] ? bad_page+0xcc/0x120 Jun 5 00:58:10 [81112905] ? prep_new_page+0x1a5/0x1b0 Jun 5 00:58:10 [815d755e] ? _raw_spin_lock+0xe/0x20 Jun 5 00:58:10 [a00b7391] ? test_range_bit+0x111/0x150 [btrfs] Jun 5 00:58:10 [81112b74] ? get_page_from_freelist+0x264/0x650 Jun 5 00:58:10 [a0073cce] ? generic_bin_search.clone.42+0x19e/0x200 [btrfs] Jun 5 00:58:10 [81113778] ? __alloc_pages_nodemask+0x118/0x830 Jun 5 00:58:10 [a0073cce] ? generic_bin_search.clone.42+0x19e/0x200 [btrfs] Jun 5 00:58:10 [815d755e] ? _raw_spin_lock+0xe/0x20 Jun 5 00:58:10 [811541d2] ?
Re: Memory leak?
Excerpts from Stephane Chazelas's message of 2011-07-08 11:41:23 -0400: 2011-07-08 11:06:08 -0400, Chris Mason: [...] So the invalidate opcode in btrfs-fixup-0 is the big problem. We're either failing to write because we weren't able to allocate memory (and not dealing with it properly) or there is a bigger problem. Does the btrfs-fixup-0 oops come before or after the ooms? Hi Chris, thanks for looking into this. It comes long before. Hours before there's any problem. So it seems unrelated. It could be the cause of the problem. We're calling BUG() with the page locked, which means that page will never ever be freed, and since this worker thread is gone, it could be messing up various parts of the reclaim code. But, this worker thread isn't supposed to get called very often...it's really catching a corner of a corner of a strange race. So we do need to get a better understanding of why and how often. You described this workload as rsync, is there anything else running? I'd definitely try without -o compress_force. -chris -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] fs/vfs/security: pass last path component to LSM on inode creation
On Wed, Dec 08, 2010 at 02:45:27PM -0500, Eric Paris wrote: SELinux would like to implement a new labeling behavior of newly created inodes. We currently label new inodes based on the parent and the creating process. This new behavior would also take into account the name of the new object when deciding the new label. This is not the (supposed) full path, just the last component of the path. This is very useful because creating /etc/shadow is different than creating /etc/passwd but the kernel hooks are unable to differentiate these operations. We currently require that userspace realize it is doing some difficult operation like that and than userspace jumps through SELinux hoops to get things set up correctly. This patch does not implement new behavior, that is obviously contained in a seperate SELinux patch, but it does pass the needed name down to the correct LSM hook. If no such name exists it is fine to pass NULL. -ETOOFUCKINGUGLY... -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Memory leak?
Excerpts from Stephane Chazelas's message of 2011-07-08 12:11:03 -0400: 2011-07-08 16:41:23 +0100, Stephane Chazelas: 2011-07-08 11:06:08 -0400, Chris Mason: [...] So the invalidate opcode in btrfs-fixup-0 is the big problem. We're either failing to write because we weren't able to allocate memory (and not dealing with it properly) or there is a bigger problem. Does the btrfs-fixup-0 oops come before or after the ooms? Hi Chris, thanks for looking into this. It comes long before. Hours before there's any problem. So it seems unrelated. Though every time I had the issue, there had been such an invalid opcode before. But also, I only had both the invalid opcode and memory issue when doing that rsync onto external hard drive. Please send along any oops output during the run. Only the first (earliest) oops matters. There's always only one in between two reboots. I've sent two already, but here they are: [...] I dug up the traces for before I switched to debian (thinking getting a newer kernel would improve matters) in case it helps: And: Jun 5 00:58:10 BUG: Bad page state in process rsync pfn:1bfdf Jun 5 00:58:10 page:ea61f8c8 count:0 mapcount:0 mapping: (null) index:0x2300 Jun 5 00:58:10 page flags: 0x110(dirty) Jun 5 00:58:10 Pid: 1584, comm: rsync Tainted: G D C 2.6.38-7-server #35-Ubuntu Jun 5 00:58:10 Call Trace: Ok, this one is really interesting. Did you get this after another oops or was it after a reboot? How easily can you recompile your kernel with more debugging flags? That should help narrow it down. I'm looking for CONFIG_SLAB_DEBUG (or slub) and CONFIG_DEBUG_PAGEALLOC -chris -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Memory leak?
2011-07-08 12:17:54 -0400, Chris Mason: [...] Jun 5 00:58:10 BUG: Bad page state in process rsync pfn:1bfdf Jun 5 00:58:10 page:ea61f8c8 count:0 mapcount:0 mapping: (null) index:0x2300 Jun 5 00:58:10 page flags: 0x110(dirty) Jun 5 00:58:10 Pid: 1584, comm: rsync Tainted: G D C 2.6.38-7-server #35-Ubuntu Jun 5 00:58:10 Call Trace: Ok, this one is really interesting. Did you get this after another oops or was it after a reboot? After the oops above (a few hours after though). The two reports were together with nothing inbetween in the kern.log. That was the only occurrence though. How easily can you recompile your kernel with more debugging flags? That should help narrow it down. I'm looking for CONFIG_SLAB_DEBUG (or slub) and CONFIG_DEBUG_PAGEALLOC [...] I can try next week. -- Stephane -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Memory leak?
2011-07-08 12:15:08 -0400, Chris Mason: [...] You described this workload as rsync, is there anything else running? [...] Nope. Nothing else. And at least initially, that was onto an empty drive so basic copy. rsync --archive --xattrs --hard-links --numeric-ids --sparse --acls Cheers, Stephane -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[GIT PULL] btrfs fixes
Hi everyone, The for-linus branch of the btrfs-unstable repo: git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git for-linus Has three more fixes. We're fixing oopsen during space balancing (btrfs filesystem balance /mnt)) and during device removal. Dave Sterba also sent in a patch to make /proc/mounts properly match a few new mount options, which he (correctly I think) considers a regression fix because it makes it hard for testers/users to verify the options in a running config. Josef Bacik (1) commits (+2/-1): Btrfs: don't panic if we get an error while balancing V2 David Sterba (1) commits (+11/-0): btrfs: add missing options displayed in mount output Miao Xie (1) commits (+7/-5): btrfs: fix oops when doing space balance Total: (3) commits (+20/-6) fs/btrfs/ctree.h |5 + fs/btrfs/inode.c | 12 +++- fs/btrfs/super.c |6 ++ fs/btrfs/volumes.c |3 ++- 4 files changed, 20 insertions(+), 6 deletions(-) -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Memory leak?
2011-07-08 12:15:08 -0400, Chris Mason: [...] I'd definitely try without -o compress_force. [...] Just started that over the night. I'm running a dstat -df at the same time and I'm seeing substantive amount of disk writes on the disks that hold the source FS (and I'm rsyncing from read-only snapshot subvolumes in case you're thinking of atimes) almost more than onto the drive holding the target FS!? --dsk/sda-- --dsk/sdb-- --dsk/sdc-- --dsk/sdd-- read writ: read writ: read writ: read writ 1000k0 : 580k0 :2176k0 : 0 0 1192k0 : 76k0 : 988k0 : 0 0 436k0 : 364k0 :1984k0 : 033M 824k0 : 812k0 :4276k0 : 0 0 3004k0 :2868k0 :5488k0 : 0 0 796k 564k: 640k 25M:2284k 4600k: 0 0 108k0 : 68k 23M: 648k 35M: 0 0 1712k 12k:1644k 12k:2476k 7864k: 0 0 732k0 : 620k0 :3192k0 : 0 0 1148k0 :1116k0 :3532k0 : 019M 1392k0 :1380k0 :4416k0 : 0 7056k 1336k0 :1212k0 :6664k0 : 0 3148k 820k0 : 784k0 :4528k0 : 048M 1336k0 :1340k0 :3964k0 : 0 8996k 1528k0 :1280k0 :2908k0 : 0 0 1352k0 :1216k0 :3880k0 : 0 0 864k0 : 888k0 :1684k0 : 0 0 1268k0 :1208k0 :3072k0 : 0 0 (source FS is on sda4+sdb+sdc, target on sdd, sda alsa has an ext4 FS) How can that be? Some garbage collection, background defrag or something like that? But then, if I stop the rsync, those writes stop as well. -- Stephane -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Memory leak?
Excerpts from Stephane Chazelas's message of 2011-07-08 16:04:12 -0400: 2011-07-08 12:15:08 -0400, Chris Mason: [...] I'd definitely try without -o compress_force. [...] Just started that over the night. I'm running a dstat -df at the same time and I'm seeing substantive amount of disk writes on the disks that hold the source FS (and I'm rsyncing from read-only snapshot subvolumes in case you're thinking of atimes) almost more than onto the drive holding the target FS!? These are probably atime updates. You can completely disable them with mount -o noatime. -chris -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Raid root filesystem?
Hi there... A year ago a btrfs raid needs to be scanned before it can be mounted. Is this still the case? I would like to have a btrfs raid as root, but so far this was not possible due to the scanning thing, and thus I still need a /boot filesystem. It would be awesome to be able to just boot a btrfs raid without the need for pesky partitions, as this would allow a system to be grown later when space gets scarce... -- -Evert- -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
