Kernel-failure - Computer freezes

Got this Kernel-failure today while making a btrfs-snapshot. # uname -r 4.2.0-1-amd64 Dez 16 20:33:14 aldebaran kernel: btrfs: page allocation failure: order:1, mode:0x204020 Dez 16 20:33:14 aldebaran kernel: CPU: 1 PID: 8016 Comm: btrfs Tainted: G U W 4.2.0-1-amd64 #1 Debian 4.2.6-3

Re: [GIT PULL] Btrfs fixes for 4.4

On Thu, Dec 10, 2015 at 11:44:50AM +, fdman...@kernel.org wrote: > From: Filipe Manana > > Hi Chris, > > Please consider the following fixes for kernel 4.4. Two of them are fixes > to new issues introduced in the 4.4 merge window and 4.4 release candidates. > The other

Re: [auto-]defrag, nodatacow - general suggestions?(was: btrfs: poor performance on deleting many large files?)

On Wed, 2015-12-09 at 16:36 +, Duncan wrote: > But... as I've pointed out in other replies, in many cases including > this > specific one (bittorrent), applications have already had to develop > their > own integrity management features Well let's move discussion upon that into the "dear

Re: btrfs: poor performance on deleting many large files

On Sun, 2015-12-13 at 07:10 +, Duncan wrote: > > So you basically mean that ro snapshots won't have their atime > > updated > > even without noatime? > > Well I guess that was anyway the recent behaviour of Linux > > filesystems, > > and only very old UNIX systems updated the atime even when

Re: [auto-]defrag, nodatacow - general suggestions?(was: btrfs: poor performance on deleting many large files?)

On Mon, 2015-12-14 at 10:51 +, Duncan wrote: > > AFAIU, the one the get's fragmented then is the snapshot, right, > > and the > > "original" will stay in place where it was? (Which is of course > > good, > > because one probably marked it nodatacow, to avoid that > > fragmentation > > problem

Re: [auto-]defrag, nodatacow - general suggestions?(was: btrfs: poor performance on deleting many large files?)

Am Wed, 9 Dec 2015 13:36:01 + (UTC) schrieb Duncan <1i5t5.dun...@cox.net>: > >> > 4) Duncan mentioned that defrag (and I guess that's also for > >> > auto- defrag) isn't ref-link aware... > >> > Isn't that somehow a complete showstopper? > > >> It is, but the one attempt at dealing with it

Ideas on unified real-ro mount option across all filesystems

Hi, In a recent btrfs patch, it is going to add a mount option to disable log replay for btrfs, just like "norecovery" for ext4/xfs. But in the discussion on the mount option name and use case, it seems better to have an unified and fs independent mount option alias for real RO mount

Re: Ideas on unified real-ro mount option across all filesystems

And here is the existing discussion in btrfs mail list, just for reference: http://thread.gmane.org/gmane.comp.file-systems.btrfs/51098 Thanks, Qu Qu Wenruo wrote on 2015/12/17 09:41 +0800: Hi, In a recent btrfs patch, it is going to add a mount option to disable log replay for btrfs, just

Re: attacking btrfs filesystems via UUID collisions?

Christoph Anton Mitterer posted on Wed, 16 Dec 2015 13:03:24 +0100 as excerpted: > Human readable lables are basically guaranteed to collide, Heh, not here, tho one could argue that my labels aren't "human readable", I suppose. grep LABEL= /etc/fstab | cut -f1 LABEL=bt0238gcn1+35l0

Re: [auto-]defrag, nodatacow - general suggestions?(was: btrfs: poor performance on deleting many large files?)

Christoph Anton Mitterer posted on Wed, 16 Dec 2015 22:59:01 +0100 as excerpted: >> And there very well might be such a tool... five or ten years down the >> road when btrfs is much more mature and generally stabilized, well >> beyond the "still maturing and stabilizing" status of the moment. >

Re: [auto-]defrag, nodatacow - general suggestions?(was: btrfs: poor performance on deleting many large files?)

Christoph Anton Mitterer posted on Wed, 16 Dec 2015 22:59:01 +0100 as excerpted: > In kinda curios, what free space fragmentation actually means here. > > Ist simply like this: > +--+-+---++ > |     F    |  D  | F |    D   | > +--+-+---++ > Where D is data

Re: [auto-]defrag, nodatacow - general suggestions?(was: btrfs: poor performance on deleting many large files?)

Christoph Anton Mitterer posted on Wed, 16 Dec 2015 22:59:01 +0100 as excerpted: > I'm a bit unsure how to read filefrag's output... (even in the > uncompressed case). > What would it show me if there was fragmentation /path/to/file: 18 extents found It tells you the number of extents found.

Re: [auto-]defrag, nodatacow - general suggestions?(was: btrfs: poor performance on deleting many large files?)

Christoph Anton Mitterer posted on Wed, 16 Dec 2015 22:59:01 +0100 as excerpted: >> he obviously didn't think thru the fact that compression MUST be a >> rewrite, thereby breaking snapshot reflinks, even were normal >> non-compression defrag to be snapshot aware, because compression >>

Re: [auto-]defrag, nodatacow - general suggestions?(was: btrfs: poor performance on deleting many large files?)

Christoph Anton Mitterer posted on Wed, 16 Dec 2015 22:59:01 +0100 as excerpted: >> It's certainly in quite a few on-list posts over the years > okay,.. in other words: no ;-) > scatter over the years list posts don't count as documentation :P =:^) -- Duncan - List replies preferred. No

Re: [PATCH v3] btrfs: Introduce new mount option to disable tree log replay

On Thu, 2015-12-17 at 01:09 +, Duncan wrote: > Well, "don't load the journal on mounting" is exactly what the option > would do.  The journal (aka log) of course has a slightly different > meaning, it's only the fsync log, but loading it is exactly what the > option would prevent, here.

Ideas to do custom operation just after mount?

Hi, Will xfstests provides some API to do some operation just after mounting a filesystem? Some filesystem(OK, btrfs again) has some function(now qgroup only) which needed to be enabled by ioctl instead of mount option. Currently, for btrfs qgroup we added special test case enabling qgroup

Re: attacking btrfs filesystems via UUID collisions?

Christoph Anton Mitterer posted on Wed, 16 Dec 2015 16:04:03 +0100 as excerpted: > On Wed, 2015-12-16 at 09:41 -0500, Chris Mason wrote: >> Hugo is right here.  reiserfs had tools that would scan and entire >> block device for metadata blocks and try to reconstruct the filesystem >> based on

Re: Ideas on unified real-ro mount option across all filesystems

On Wed, Dec 16, 2015 at 09:15:59PM -0600, Eric Sandeen wrote: > > > On 12/16/15 7:41 PM, Qu Wenruo wrote: > > Hi, > > > > In a recent btrfs patch, it is going to add a mount option to disable > > log replay for btrfs, just like "norecovery" for ext4/xfs. > > > > But in the discussion on the

Re: [auto-]defrag, nodatacow - general suggestions?(was: btrfs: poor performance on deleting many large files?)

Christoph Anton Mitterer posted on Wed, 16 Dec 2015 22:59:01 +0100 as excerpted: > On Wed, 2015-12-09 at 16:36 +, Duncan wrote: >> But... as I've pointed out in other replies, in many cases including >> this specific one (bittorrent), applications have already had to >> develop their own

Re: Ideas on unified real-ro mount option across all filesystems

On 12/16/15 7:41 PM, Qu Wenruo wrote: > Hi, > > In a recent btrfs patch, it is going to add a mount option to disable > log replay for btrfs, just like "norecovery" for ext4/xfs. > > But in the discussion on the mount option name and use case, it seems > better to have an unified and fs

Re: [PATCH v3] btrfs: Introduce new mount option to disable tree log replay

Christoph Anton Mitterer posted on Wed, 16 Dec 2015 12:45:00 +0100 as excerpted: > On Wed, 2015-12-16 at 11:10 +, Duncan wrote: >> And noload doesn't have the namespace collision problem norecovery does >> on btrfs, so I'd strongly suggest using it, at least as an alias for >> whatever other

Re: dear developers, can we have notdatacow + checksumming, plz?

On Tue, 2015-12-15 at 11:00 -0500, Austin S. Hemmelgarn wrote: > > Well sure, I think we'de done most of this and have dedicated > > controllers, at least of a quality that funding allows us ;-) > > But regardless how much one tunes, and how good the hardware is. If > > you'd then loose always a

Re: [PATCH v3] btrfs: Introduce new mount option to disable tree log replay

Qu Wenruo posted on Wed, 16 Dec 2015 09:36:23 +0800 as excerpted: > David Sterba wrote on 2015/12/14 18:32 +0100: >> On Thu, Dec 10, 2015 at 10:34:06AM +0800, Qu Wenruo wrote: >>> Introduce a new mount option "nologreplay" to co-operate with "ro" >>> mount option to get real readonly mount, like

Re: [PATCH v3] btrfs: Introduce new mount option to disable tree log replay

On Wed, 2015-12-16 at 11:10 +, Duncan wrote: > And noload doesn't have the namespace collision problem norecovery > does > on btrfs, so I'd strongly suggest using it, at least as an alias for > whatever other btrfs-specific name we might choose. but noload is, AFAIU, not what's desired

Re: attacking btrfs filesystems via UUID collisions?

On Tue, 2015-12-15 at 08:54 -0500, Austin S. Hemmelgarn wrote: > Except for one thing:  Automobiles actually provide a measurable > significant benefit to society.  What specific benefit does embedding > the filesystem UUID in the metadata actually provide? I guess that's quite obvious. You want

Re: attacking btrfs filesystems via UUID collisions?

On Tue, 2015-12-15 at 14:18 +, Hugo Mills wrote: >    That one's easy to answer. It deals with a major issue that > reiserfs had: if you have a filesystem with another filesystem image > stored on it, reiserfsck could end up deciding that both the metadata > blocks of the main filesystem *and*

Re: attacking btrfs filesystems via UUID collisions?

On Tue, 2015-12-15 at 11:03 -0500, Austin S. Hemmelgarn wrote: > May I propose the alternative option of adding a flag to tell mount > to > _only_ use the devices specified in the options? That's one part of exactly what I propose since a few days :-P (no one seems to read my mails ;-) ) Plus

Re: [PATCH v3] btrfs: Introduce new mount option to disable tree log replay

On 2015-12-16 06:10, Duncan wrote: Qu Wenruo posted on Wed, 16 Dec 2015 09:36:23 +0800 as excerpted: David Sterba wrote on 2015/12/14 18:32 +0100: On Thu, Dec 10, 2015 at 10:34:06AM +0800, Qu Wenruo wrote: Introduce a new mount option "nologreplay" to co-operate with "ro" mount option to get

Re: [PATCH v3] btrfs: Introduce new mount option to disable tree log replay

On Mon, Dec 14, 2015 at 12:50:37PM -0500, Austin S. Hemmelgarn wrote: > On 2015-12-14 12:32, David Sterba wrote: > > On Thu, Dec 10, 2015 at 10:34:06AM +0800, Qu Wenruo wrote: > >> Introduce a new mount option "nologreplay" to co-operate with "ro" mount > >> option to get real readonly mount, like

Re: need to recover large file

Langhorst, Brad posted on Wed, 16 Dec 2015 03:13:48 + as excerpted: > Hi: > > I just screwed up
 spent the last 3 weeks generting a 400G file > (genome assembly) . > Went to back it up and swapped the arguments to tar (tar Jcf my_precious > my_precious.tar.xz) > what was once 400G is now

Re: [PATCH v3] btrfs: Introduce new mount option to disable tree log replay

On 2015-12-16 07:34, Christoph Anton Mitterer wrote: On Wed, 2015-12-16 at 07:12 -0500, Austin S. Hemmelgarn wrote: I kind of agree with Christoph here. I don't think that noload should be the what we actually use, although I do think having it as an alias for whatever name we end up using

Re: [PATCH v3] btrfs: Introduce new mount option to disable tree log replay

On Wed, Dec 16, 2015 at 09:36:23AM +0800, Qu Wenruo wrote: > > > > mount -o ro,nowr /dev/sdx /mnt > > > > would work when switching kernels. > > > That would be nice. > > I'd like to forward the idea/discussion to filesystem ml, not only btrfs > maillist. Good idea. > Such behavior should

Re: Still not production ready

On Tue, Dec 15, 2015 at 06:30:58PM -0800, Liu Bo wrote: > On Wed, Dec 16, 2015 at 10:19:00AM +0800, Qu Wenruo wrote: > > >max_stripe_size is fixed at 1GB and the chunk size is stripe_size * > > >data_stripes, > > >may I know how your partition gets a 10GB chunk? > > > > Oh, it seems that I

Re: attacking btrfs filesystems via UUID collisions?

On Wed, Dec 16, 2015 at 01:03:38PM +0100, Christoph Anton Mitterer wrote: > On Tue, 2015-12-15 at 14:18 +, Hugo Mills wrote: > >    That one's easy to answer. It deals with a major issue that > > reiserfs had: if you have a filesystem with another filesystem image > > stored on it, reiserfsck

Re: attacking btrfs filesystems via UUID collisions?

On Wed, 2015-12-16 at 09:41 -0500, Chris Mason wrote: > Hugo is right here.  reiserfs had tools that would scan and entire > block > device for metadata blocks and try to reconstruct the filesystem > based > on what it found. Creepy... at least when talking about a "normal" fsck... good that btrfs

Re: attacking btrfs filesystems via UUID collisions?

On Tue, 2015-12-15 at 14:42 +, Hugo Mills wrote: >    I would suggest trying to migrate to a state where detecting more > than one device with the same UUID and devid is cause to prevent the > FS from mounting, unless there's also a "mount_duplicates_yes_i_ >

Re: [PATCH v3] btrfs: Introduce new mount option to disable tree log replay

On Wed, 2015-12-16 at 07:12 -0500, Austin S. Hemmelgarn wrote: > I kind of agree with Christoph here.  I don't think that noload > should > be the what we actually use, although I do think having it as an > alias > for whatever name we end up using would be a good thing. No, because people would

Re: attacking btrfs filesystems via UUID collisions?

On Tue, 2015-12-15 at 09:19 -0500, Austin S. Hemmelgarn wrote: > Um, no you don't have direct physical access to the hardware with an > ATM, at least, not unless you are going to take apart the cover and > anything else in your way (and probably set off internal alarms). Well access to the

Re: [PATCH v3] btrfs: Introduce new mount option to disable tree log replay

On Wed, 2015-12-16 at 07:57 -0500, Austin S. Hemmelgarn wrote: > No, because we should ease the transition from other filesystems to > the > greatest extent reasonably possible.  It should be clearly documented > as > an alias for compatibility with ext{3,4}, and that it might go away > in >

Re: dear developers, can we have notdatacow + checksumming, plz?

Austin S. Hemmelgarn posted on Tue, 15 Dec 2015 11:00:40 -0500 as excerpted: > And in particular, the only > journaling filesystem that I know of that even allows the option of > journaling the file contents instead of just metadata is ext4. IIRC, ext3 was the first to have it in Linux mainline,

Re: btrfs: poor performance on deleting many large files

Lionel Bouton posted on Tue, 15 Dec 2015 03:38:33 +0100 as excerpted: > I just checked: this has only be made crystal-clear in the latest > man-pages version 4.03 released 10 days ago. > > The mount(8) page of Gentoo's current stable man-pages (4.02 release in > August) which is installed on my

Re: dear developers, can we have notdatacow + checksumming, plz?

Austin S. Hemmelgarn posted on Tue, 15 Dec 2015 11:00:40 -0500 as excerpted: > AFAIUI, checksums are stored per-instance for every block. This is > important in a multi-device filesystem in case you lose a device, so > that you still have a checksum for the block. There should be no >