Re: [PATCH v3] btrfs: Introduce new mount option to disable tree log replay

On Wed, 2015-12-16 at 09:36 +0800, Qu Wenruo wrote: > One sad example is, we can't use 'norecovery' mount option to disable > log replay in btrfs, as there is 'recovery' mount option already. I think "norecovery" would anyway not really fit... the name should rather indicated, that from the

Re: attacking btrfs filesystems via UUID collisions?

On Mon, 2015-12-14 at 14:26 -0700, Chris Murphy wrote: > The automobile is invented and due to the ensuing chaos, common > practice of doing whatever the F you wanted came to an end in favor > of > rules of the road and traffic lights. I'm sure some people went > ballistic, but for the most part

Re: dear developers, can we have notdatacow + checksumming, plz?

On Mon, 2015-12-14 at 17:42 +1100, Russell Coker wrote: > My understanding of BTRFS is that the metadata referencing data > blocks has the > checksums for those blocks, then the blocks which link to that > metadata (EG > directory entries referencing file metadata) has checksums of those. You

Re: [PATCH v3] btrfs: Introduce new mount option to disable tree log replay

On Mon, 2015-12-14 at 15:20 -0500, Austin S. Hemmelgarn wrote: > On 2015-12-14 14:44, Christoph Anton Mitterer wrote: > > On Mon, 2015-12-14 at 14:33 -0500, Austin S. Hemmelgarn wrote: > > > The traditional reasoning was that read-only meant that users > > > cou

Re: attacking btrfs filesystems via UUID collisions?

On Mon, 2015-12-14 at 13:55 -0700, Chris Murphy wrote: > I'm aware of this proof of concept. I'd put it, and this one, in the > realm of a targeted attack, so it's not nearly as likely as other > problems needing fixing. That doesn't mean don't understand it better > so it can be fixed. It means

Re: attacking btrfs filesystems via UUID collisions?

On Mon, 2015-12-14 at 08:23 -0500, Austin S. Hemmelgarn wrote: > The reason that this isn't quite as high of a concern is because > performing this attack requires either root access, or direct > physical > access to the hardware, and in either case, your system is already > compromised. No

Re: dear developers, can we have notdatacow + checksumming, plz?

On Mon, 2015-12-14 at 09:16 -0500, Austin S. Hemmelgarn wrote: > > When one starts to get a bit deeper into btrfs (from the admin/end- > > user > > side) one sooner or later stumbles across the recommendation/need > > to > > use nodatacow for certain types of data (DBs, VM images, etc.) and > >

Re: btrfs: poor performance on deleting many large files

On Mon, 2015-12-14 at 09:24 -0500, Austin S. Hemmelgarn wrote: > Unless things have changed very recently, even many modern systems > update atime on read-only filesystems, unless the media itself is > read-only. Seriously? Oh... *sigh*... You mean as in Linux, ext*, xfs? > If you have software

Re: [PATCH v3] btrfs: Introduce new mount option to disable tree log replay

On Mon, 2015-12-14 at 18:32 +0100, David Sterba wrote: > I've read the discussions around the change and from the user's POV > I'd > suggest to add another mount option that would be just an alias for > any > mount options that would implement the 'hard-ro' semantics. Nice to hear...  > Say it's

Re: [PATCH v3] btrfs: Introduce new mount option to disable tree log replay

On Mon, 2015-12-14 at 12:50 -0500, Austin S. Hemmelgarn wrote: > It should also imply noatime.  I'm not sure how BTRFS handles atime > when > mounted RO, but I know a lot of old UNIX systems updated atime even > on > filesystems mounted RO, and I know that at least at one point Linux > did too.

Re: [PATCH v3] btrfs: Introduce new mount option to disable tree log replay

On Mon, 2015-12-14 at 14:33 -0500, Austin S. Hemmelgarn wrote: > The traditional reasoning was that read-only meant that users > couldn't > change anything Where I'd however count the atime changes to. The atimes wouldn't change magically, but only because the user stared some program, configured

Re: btrfs: poor performance on deleting many large files

On Mon, 2015-12-14 at 15:27 -0500, Austin S. Hemmelgarn wrote: > On 2015-12-14 14:39, Christoph Anton Mitterer wrote: > > On Mon, 2015-12-14 at 09:24 -0500, Austin S. Hemmelgarn wrote: > > > Unless things have changed very recently, even many modern > > > systems >

project idea: per-object default mount-options / more btrfs-properties / chattr attributes (was: btrfs: poor performance on deleting many large files)

Just FYI: On Mon, 2015-12-14 at 15:27 -0500, Austin S. Hemmelgarn wrote: > > My idea would be basically, that having a noatime btrfs-property, > > which > > is perhaps even set automatically, would be an elegant way of doing > > that. > > I just haven't had time to properly write that up and add

Re: btrfs: poor performance on deleting many large files

On Mon, 2015-12-14 at 22:30 +0100, Lionel Bouton wrote: > Mutt is often used as an example but tmpwatch uses atime by default > too > and it's quite useful. Hmm one could probably argue that these few cases justify the use of separate filesystems (or btrfs subvols ;) ), so that the majority could

Re: [auto-]defrag, nodatacow - general suggestions?(was: btrfs: poor performance on deleting many large files?)

On Wed, 2015-12-09 at 13:36 +, Duncan wrote: > Answering the BTW first, not to my knowledge, and I'd be > skeptical.  In > general, btrfs is cowed, and that's the focus.  To the extent that > nocow > is necessary for fragmentation/performance reasons, etc, the idea is > to > try to make cow

Re: attacking btrfs filesystems via UUID collisions?

On Fri, 2015-12-11 at 16:06 -0700, Chris Murphy wrote: > For anything but a new and empty Btrfs volume What's the influence of the fs being new/empty? > this hypothetical > attack would be a ton easier to do on LVM and mdadm raid because they > have a tiny amount of metadata to spoof compared to

Re: attacking btrfs filesystems via UUID collisions?

On Sat, 2015-12-12 at 02:34 +0100, S.J. wrote: > A bit more about the dd-is-bad-topic: > > IMHO it doesn't matter at all. Yes, fully agree. > a) For this specific problem here, fixing a security problem > automatically > fixes the risk of data corruption because careless cloning+mounting >

Re: [auto-]defrag, nodatacow - general suggestions?(was: btrfs: poor performance on deleting many large files?)

Two more on these: On Thu, 2015-11-26 at 00:33 +, Hugo Mills wrote: > 3) When I would actually disable datacow for e.g. a subvolume that > > holds VMs or DBs... what are all the implications? > > Obviously no checksumming, but what happens if I snapshot such a > > subvolume or if I

dear developers, can we have notdatacow + checksumming, plz?

(consider that question being asked with that face on: http://goo.gl/LQaOuA) Hey. I've had some discussions on the list these days about not having checksumming with nodatacow (mostly with Hugo and Duncan). They both basically told me it wouldn't be straight possible with CoW, and Duncan thinks

Re: subvols and parents - how?

On Wed, 2015-12-09 at 10:53 +, Duncan wrote: > If you use the recipe (subvol create, cp with reflink) it suggests > there, > you'll end up with the reflinked copy in a subvol. > > You can then mount that subvol over top of the existing dir, and > *new* > file opens will access the new

Re: subvols, ro- and bind mounts - how?

On Thu, 2015-12-10 at 19:32 -0700, Chris Murphy wrote: > That seems due for a revision because I do rw, ro, rw, rw, ro mounts > in sequence and they stick fine. In fact they stick with the same > subvolume. > > [root@f23m ]# mount /dev/sda7 /mnt/1 -o subvol=home > [root@f23m ]# mount /dev/sda7

Re: Will "btrfs check --repair" fix the mounting problem?

On Sat, 2015-12-12 at 13:16 -0700, Chris Murphy wrote: > > What is the better way to get data? send/receive works only with RO > > snapshots. Is there another way to preserve subvolumes and CoW > > structure (a lot of files was copied between subvols using "cp > > --reflink=always")? Or just

Re: btrfs: poor performance on deleting many large files

On Sat, 2015-11-28 at 06:49 +, Duncan wrote: > Christoph Anton Mitterer posted on Sat, 28 Nov 2015 04:57:05 +0100 as > excerpted: > > Still, specifically for snapshots that's a bit unhandy, as one > > typically > > doesn't mount each of them... one rather mount e.g.

Re: attacking btrfs filesystems via UUID collisions?

Sorry, I'm just about to change my mail system, and used a bogus test From: address in the previous mail (please replace fo@fo with cales...@scientia.net). Apologies for any inconveniences and this noise here. Cheers, Chris. smime.p7s Description: S/MIME cryptographic signature

Re: attacking btrfs filesystems via UUID collisions?

On Thu, 2015-12-10 at 12:42 -0700, Chris Murphy wrote: > That isn't what I'm suggesting. In the multiple device volume case > where there are two exact (same UUID, same devid, same generation) > instances of one of the block devices, Btrfs could randomly choose > either one if it's an RO mount.

Re: attacking btrfs filesystems via UUID collisions?

On Wed, 2015-12-09 at 22:48 +0100, S.J. wrote: > > 3. Some way to fail gracefully, when there's ambiguity that cannot > > be > > resolved. Once there are duplicate devs (dd or lvm snapshots, etc) > > then there's simply no way to resolve the ambiguity automatically, > > and > > the volume should

Re: subvols and parents - how?

On Sat, 2015-12-12 at 03:32 +0100, Christoph Anton Mitterer wrote: > What's still missing now, IMHO, is: > - a guide when one should make subvole (e.g. keeping things on the > root > fs together, unless it's separate like /var/www is usually, but > /var/lib typically "cor

Re: subvols, ro- and bind mounts - how?

Hey. I'd have an additional question about subvols O:-) Given the following setup: 5 | +--root (subvol, /)    +-- mnt (dir) with the following done: - init 1 - remount,ro / (i.e. the subvol root) - mount /dev/btrfs-device /mnt (i.e. mount the top subvol at /mnt) The following happened: - / was

Re: subvols, ro- and bind mounts - how?

On Thu, 2015-12-10 at 23:36 +0100, S.J. wrote: > Quote: > > " Most mount options apply to the whole filesystem, and only the > options > for the first subvolume > to be mounted will take effect. This is due to lack of implementation > and may change in the future. " > > from

Re: [PATCH] btrfs: Introduce new mount option to disable tree log replay

On Tue, 2015-12-08 at 07:15 -0500, Austin S Hemmelgarn wrote: > Despite this, it really isn't a widely known or well documented > behavior > outside of developers, forensic specialists, and people who have had > to > deal with the implications it has on data recovery.  There really > isn't >

Re: [RFC] Btrfs device and pool management (wip)

On Mon, 2015-11-30 at 13:17 -0700, Chris Murphy wrote: > On Mon, Nov 30, 2015 at 7:51 AM, Austin S Hemmelgarn > wrote: > > > General thoughts on this: > > 1. If there's a write error, we fail unconditionally right now.  It > > would be > > nice to have a configurable number

Re: [auto-]defrag, nodatacow - general suggestions?(was: btrfs: poor performance on deleting many large files?)

Hey Hugo, On Thu, 2015-11-26 at 00:33 +, Hugo Mills wrote: >    Answering the second part first, no, it can't. Thanks so far :) >    The issue is that nodatacow bypasses the transactional nature of > the FS, making changes to live data immediately. This then means that > if you modify a

Re: [auto-]defrag, nodatacow - general suggestions?(was: btrfs: poor performance on deleting many large files?)

On 2015-11-27 00:08, Duncan wrote: > Christoph Anton Mitterer posted on Thu, 26 Nov 2015 01:23:59 +0100 as > excerpted: >> 1) AFAIU, the fragmentation problem exists especially for those files >> that see many random writes, especially, but not limited to, big files. >> No

Re: subvols and parents - how?

On Fri, 2015-11-27 at 02:02 +, Duncan wrote: > Uhm, I don't get the big security advantage here... whether nested > > or > > manually mounted to a subdir,... if the permissions are insecure > > I'll > > have a problem... if they're secure, than not. > Consider a setuid-root binary with a

Re: attacking btrfs filesystems via UUID collisions?

On Sun, 2015-12-06 at 22:34 +0800, Qu Wenruo wrote: > Not sure about LVM/MD, but they should suffer the same UUID conflict > problem. Well I had that actually quite often in LVM (i.e. same UUIDs visible on the same system), basically because we made clones from one template VM image and when that

Re: kernel call trace during send/receive

Hey. Hmm I guess no one has any clue about that error? Well it seems at least that an fsck over the receiving fs passes through without any error. Cheers, Chris. On Fri, 2015-11-27 at 02:49 +0100, Christoph Anton Mitterer wrote: > Hey. > > Just got the following during send/receiv

Re: attacking btrfs filesystems via UUID collisions? (was: Subvolume UUID, data corruption?)

On Sun, 2015-12-06 at 04:06 +, Duncan wrote: > There's actually a number of USB-based hardware and software vulns > out > there, from the under $10 common-component-capacitor-based charge- > and-zap > (charges off the 5V USB line, zaps the port with several hundred > volts > reverse-polarity,

Re: subvols and parents - how?

On Fri, 2015-11-27 at 01:02 +, Duncan wrote: [snip snap] > #1 could be a pain to setup if you weren't actually mounting it > previously, just relying on the nested tree, AND... > > #2 The point I was trying to make, now, to mount it you'll mount not > a > native nested subvol, and not a

Re: [PATCH] btrfs: Introduce new mount option to disable tree log replay

On Mon, 2015-12-07 at 11:29 -0600, Eric Sandeen wrote: > FWIW, new mount options and their descriptions should be added to > BTRFS-MOUNT(5) > as well. Also, from the end-user perspective, there should be: 1) another option like (hard-ro) which is defined to imply any other options that are

Re: [PATCH] btrfs: Introduce new mount option to disable tree log replay

On Mon, 2015-12-07 at 17:06 -0600, Eric Sandeen wrote: > Yeah, I don't know that this is true.  It hasn't been true for over a > decade (2?), with the most widely-used filesystem in linux history, > i.e. > ext3. Based on what? I'd now many sysadmins who don't expect that e.g. the journal is

Re: attacking btrfs filesystems via UUID collisions? (was: Subvolume UUID, data corruption?)

On Sat, 2015-12-05 at 13:19 +, Duncan wrote: > The problem with btrfs is that because (unlike traditional > filesystems) > it's multi-device, it needs some way to identify what devices belong > to a > particular filesystem. Sure, but that applies to lvm, or MD as well... and I wouldn't know

Re: attacking btrfs filesystems via UUID collisions? (was: Subvolume UUID, data corruption?)

On Sat, 2015-12-05 at 12:01 +, Hugo Mills wrote: > On Sat, Dec 05, 2015 at 04:28:24AM +0100, Christoph Anton Mitterer > wrote: > > On Fri, 2015-12-04 at 13:07 +, Hugo Mills wrote: > > > I don't think it'll cause problems. > > Is there any guaranteed behaviou

Re: Subvolume UUID, data corruption?

On Fri, 2015-12-04 at 13:07 +, Hugo Mills wrote: > I don't think it'll cause problems. Is there any guaranteed behaviour when btrfs encounters two filesystems (i.e. not talking about the subvols now) with the same UUID? Given that it's long standing behaviour that people could clone

Re: attacking btrfs filesystems via UUID collisions? (was: Subvolume UUID, data corruption?)

Thinking a bit more I that, I came to the conclusion that it's actually security relevant that btrfs deals gracefully with filesystems having the same UUID: Getting to know someone else's filesystem's UUID may be more easily possible than one may think. It's usually not considered secret and

Re: btrfs crashing the kernel with Seagate 8TB SMR drives.

Any chances that this is: https://bugzilla.kernel.org/show_bug.cgi?id=93581 Cheers, Chris. smime.p7s Description: S/MIME cryptographic signature

Re: slowness when cp respectively send/receiving on top of dm-crypt

On Sat, 2015-11-28 at 11:34 -0700, Chris Murphy wrote: > It sounds to me like maybe LUKS is configured to use an encryption > algorithm that isn't subject to CPU optimized support, e.g. aes-xts > on > my laptop gets 1600MiB/s where serpent-cbc gets only 68MiB/s and pegs > the CPU. This is reported

Re: How to detect / notify when a raid drive fails?

On Fri, 2015-11-27 at 17:16 +0800, Anand Jain wrote: >   I understand as a user, a full md/lvm set of features are important >   to begin operations using btrfs and we don't have it yet. I have to >   blame it on the priority list. What's would be especially nice from the admin side, would be

slowness when cp respectively send/receiving on top of dm-crypt

Hey. Not sure if that's valuable input for the devs, but here's some vague real-world report about performance: I'm just copying (via send/receive) a large filesystem (~7TB) from on HDD over to another. The devices are both connected via USB3, and each of the btrfs is on top of dm-crypt. It's

Re: slowness when cp respectively send/receiving on top of dm-crypt

Hey. Send/receiving the master to the backup has finished just before... and now - not that I wouldn't trust btrfs, the hardware, etc. - I ran a complete diff --recursive --no-dereference over the snapshots on the two disks. The two btrfs are mounted ro (thus no write IO), there is not really

Re: btrfs: poor performance on deleting many large files

On Fri, 2015-11-27 at 03:38 +, Duncan wrote: > AFAIK, per-subvolume *atime mounts should already be working. Ah I see. :) Still, specifically for snapshots that's a bit unhandy, as one typically doesn't mount each of them... one rather mount e.g. the top level subvol and has a subdir

cannot move ro-snapshot directly but indirectly works

Hey. Not sure whether this is intended or not, but it feels at least somewhat strange: Consider I have a readonly snapshot (the only subvol here): /btrfs/snapshots/ro-snapshot now I want to move it to the dir: /btrfs/snapshots/foo/ i.e.  mv /btrfs/snapshots/ro-snapshot /btrfs/snapshots/foo/ but

Re: slowness when cp respectively send/receiving on top of dm-crypt

On Fri, 2015-11-27 at 20:00 +0100, Henk Slager wrote: > As far as I can guess this is transfers between Seagate Archive 8TB > SMR drives. Yes it is,... and I though about SMR being the reason at first, too, but: - As far as I understood SMR, it shouldn't kick in when I do what is mostly streaming

Re: [PATCH v2] btrfs-progs: fsck: Fix a false alert where extent record has wrong metadata flag

On Fri, 2015-11-27 at 08:40 +0800, Qu Wenruo wrote: > But since there is no real error sure...  > feel free to keep using it or just re > format it with skinny-metadata. That's just onging =) Thanks for all your efforts in that issue =) Chris. smime.p7s Description: S/MIME cryptographic

Re: btrfs: poor performance on deleting many large files

On Thu, 2015-11-26 at 23:29 +, Duncan wrote: > > but only on meta-data blocks, right? > Yes. Okay... so it'll at most get the whole meta-data for a snapshot separately and not shared anymore... And when these are chained as in ZFS,.. it probably amplifies... i.e. a change deep down in the tree

kernel call trace during send/receive

Hey. Just got the following during send/receiving a big snapshot from one btrfs to another fresh one. Both under kernel 4.2.6, tools 4.3 The send/receive seems to continue however... Any ideas what that means? Cheers, Chris. Nov 27 01:52:36 heisenberg kernel: [ cut here

Re: [PATCH v2] btrfs-progs: fsck: Fix a false alert where extent record has wrong metadata flag

non-skinny-metadata filesystem. > > > > Fix it by set correct metadata value before calling > > add_extent_rec(). > > > > Reported-by: Christoph Anton Mitterer <cales...@scientia.net> > > Signed-off-by: Qu Wenruo <quwen...@cn.fujitsu.com> > > Pat

Re: btrfs: poor performance on deleting many large files

On Thu, 2015-11-26 at 16:52 +, Duncan wrote: > For people doing snapshotting in particular, atime updates can be a > big > part of the differences between snapshots, so it's particularly > important > to set noatime if you're snapshotting. What everything happens when that is left at

Re: [auto-]defrag, nodatacow - general suggestions?(was: btrfs: poor performance on deleting many large files?)

Hey. I've worried before about the topics Mitch has raised. Some questions. 1) AFAIU, the fragmentation problem exists especially for those files that see many random writes, especially, but not limited to, big files. Now that databases and VMs are affected by this, is probably broadly known in

Re: shall distros run btrfsck on boot?

On Tue, 2015-11-24 at 22:33 +, Hugo Mills wrote: > whereas a read-only mount of a journalling FS _must_ modify the disk > data after an unclean shitdown, in order to be useful (because the FS > isn't consistent without the journal replay). I've always considered that rather a bug,... or at

Re: subvols and parents - how?

On Tue, 2015-11-24 at 21:55 +, Hugo Mills wrote: >    In practice, new content is checked by a number of people when > it's > put in, so the case of someone putting random poorly-thought-out crap > in the wiki isn't particularly likely to happen. Well... it may work in 99% cases... but there

btrfs-mount improvemt suggestions

compression, useful when re-mounting), *zlib* (the default if no 'type' is set), *lzo* + Enabling compression implies the options *datacow* and *datasum*. That would also include these changes: commit 88a0ba7065e09497806ffc2a493ab72d0940e1af Author: Christoph Anton Mitterer

Re: bad extent [5993525264384, 5993525280768), type mismatch with chunk

On Wed, 2015-11-25 at 08:59 +0800, Qu Wenruo wrote: > Did you use the complied btrfsck? Or use the system btrfsck by > mistake? I'm pretty sure cause I already did the whole procedure twice, but let me repeat and record it here just to be 100% sure: $ make clean Cleaning $ md5sum cmds-check.c

Re: bad extent [5993525264384, 5993525280768), type mismatch with chunk

Hey again. So it seems that data-b is always fine (well at least three times in a row) and data-old-a always gives errors. including e.g: bad extent [3067663679488, 3067663695872), type mismatch with chunk bad extent [3067663876096, 3067663892480), type mismatch with chunk bad extent

Re: subvols and parents - how?

On Tue, 2015-11-24 at 23:30 +, Hugo Mills wrote: >    Yes, that makes sense. Feel free to shamelessly use my idea (as well as the one to call btrfs' RAID1 replica2 or something else) :-O >    With a recent mv root@heisenberg:/mnt# mv --version mv (GNU coreutils) 8.23 => not recent enough...

Re: subvols and parents - how?

On Tue, 2015-11-24 at 08:29 +, Duncan wrote: > OK, found it on the wiki.  It wasn't under use-cases, where I > initially > thought to look, but under sysadmin guide.  Specifically, see section > 4.2, managing snapshots, but I'd suggest reading the entire > subvolumes > discussion, section 4,

Re: btrfs send reproducibly fails for a specific subvolume after sending 15 GiB, scrub reports no errors

On Tue, 2015-11-24 at 15:58 -0500, Austin S Hemmelgarn wrote: > I had tried using send/receive once with -p, but had numerous issues.   > The incrementals I've been doing have used -c instead, and I hadn't had  > any issues with data loss with that.  The issue outlined here was only a  > small

Re: btrfs send reproducibly fails for a specific subvolume after sending 15 GiB, scrub reports no errors

On Tue, 2015-11-24 at 15:44 -0500, Austin S Hemmelgarn wrote: > I would say it's currently usable for one-shot stuff, but probably > not > reliably useable for automated things without some kind of > administrative oversight.  In theory, it wouldn't be hard to write a > script to automate

Re: btrfs send reproducibly fails for a specific subvolume after sending 15 GiB, scrub reports no errors

On Tue, 2015-11-24 at 21:27 +, Hugo Mills wrote: >    -p only sends the file metadata for the changes from the reference > snapshot to the sent snapshot. -c sends all the file metadata, but > will preserve the reflinks between the sent snapshot and the (one or > more) reference snapshots. Let

Re: shall distros run btrfsck on boot?

On Tue, 2015-11-24 at 11:14 -0600, Eric Sandeen wrote: > In a nutshell, though, I think a filesystem repair should be an > admin-initiated > action, not something that surprises you on a boot, at least for a > journaling > filesystem which is designed to maintain its integrity even in the > face

Re: bad extent [5993525264384, 5993525280768), type mismatch with chunk

On Tue, 2015-11-24 at 13:35 +0800, Qu Wenruo wrote: > Hopes you didn't wait too long. No worries, didn't hold my breath ;) > The fixing patch is CCed to you, or you can get it from patchwork: > https://patchwork.kernel.org/patch/7687611/ Unfortunately that doesn't make the error messages go

Re: btrfs send reproducibly fails for a specific subvolume after sending 15 GiB, scrub reports no errors

Hey. All that sounds pretty serious, doesn't it? So in other words, AFAIU, send/receive cannot really be reliably used. I did so far for making incremental backups, but I've also experienced some problems (though not what this is about here). Cheers, Chris. smime.p7s Description: S/MIME

Re: bad extent [5993525264384, 5993525280768), type mismatch with chunk

On Mon, 2015-11-23 at 09:10 +0800, Qu Wenruo wrote: > Also, you won't want compiler to do extra optimization I did the following: $ export CFLAGS="-g -O0 -Wall -D_FORTIFY_SOURCE=2" $ ./configure --disable-convert --disable-documentation So if you want me to get rid of _FORTIFY_SOURCE, please

Re: shall distros run btrfsck on boot?

On Tue, 2015-11-24 at 04:35 +, Duncan wrote: > I'm a list regular and btrfs user, not a dev, but all the indications > continue to point to _not_ running it automatically at boot, nobody > even > _suggesting_ otherwise. Sure, I just asked because maybe that would have just been an

subvols and parents - how?

Hey. I'd have a, mainly administrative, question. When I use subvolumes than these have always a parent subvolume (except ID5), so I can basically decide between two ways: a) make child subvolumes, e.g. 5 | +-root   (=subvol, mountpoint /)   +-boot/   +-root/   +-lib/   +-home/ (=subvolume) and

Re: bad extent [5993525264384, 5993525280768), type mismatch with chunk

On Tue, 2015-11-24 at 08:46 +0800, Qu Wenruo wrote: > But there are also some other places like line 4411, 4394 and 4387. Ah of course, I didn't have a look for further places $ grep -n "rec->wrong_chunk_type = 1" cmds-check.c 4387: rec->wrong_chunk_type = 1; 4394:

Re: bad extent [5993525264384, 5993525280768), type mismatch with chunk

On Tue, 2015-11-24 at 10:54 +0800, Qu Wenruo wrote: > And it would be even better if you want to be a lab mouse for > incoming fixing patches. Sure,.. if I get some cheese... and it would be great if you could give me patches that apply to 4.3. > (It won't hurt nor destroy your data) wouldn't  

Re: bad extent [5993525264384, 5993525280768), type mismatch with chunk

On Tue, 2015-11-24 at 10:09 +0800, Qu Wenruo wrote: > I'll dig further to see what's causing the problem. I guess you'd prefer if I keep the fs for later verification? > Thanks for all the debug info, it really helps a lot! Well thanks for your efforts as well :) Chris. smime.p7s Description:

shall distros run btrfsck on boot?

Hey. Short question since that came up on debian-devel. Now that btrfs check get's more and more useful, are the developers going to recommend running it periodically on boot (of course that wouldn't work right now, as it would *always* check)? Plus... is btrfs check (without any arguments)

Re: [PATCH v2 3/5] btrfs-progs: kernel based default features for mkfs

On Mon, 2015-11-23 at 11:05 -0500, Austin S Hemmelgarn wrote: > I would find it useful if btrfs gives a warning if it creates a > > filesystem which (because unsupported in the current kernel) lacks > > features which are considered default by then. > It should give a warning if the user requests

Re: [PATCH v2 3/5] btrfs-progs: kernel based default features for mkfs

Hey. On Mon, 2015-11-23 at 20:56 +0800, Anand Jain wrote: > This patch disables default features based on the running kernel. Not sure if that's very realistic in practise (most people will have some distro, whose btrfsprogs version probably matches the kernel), but purely from the end-user PoV:

Re: bad extent [5993525264384, 5993525280768), type mismatch with chunk

Hey Qu. On Sun, 2015-11-22 at 10:04 +0800, Qu Wenruo wrote: > If any of you can recompile btrfs-progs and use gdb to debug it, > would > anyone please to investigate where did the wrong_chunk_type is set? > It is in the function check_extent_type(): Not 100% what you want... AFAIU, you just

Re: bad extent [5993525264384, 5993525280768), type mismatch with chunk

On Fri, 2015-11-20 at 17:23 +0100, Laurent Bonnaud wrote: > So here is the output of "btrfs-debug-tree -t 2 " in case it may Gosh... 15M via mail?! o.O Anyway an update from my side... I've copied all data from the fs in question to a new btrfs,... done under Linux 4.2.6 and btrfs-progs v4.3. No

Re: [Enhancement] ... and please rename "raid1" to something better

On Fri, 2015-11-20 at 11:05 +, Duncan wrote: > It's missing raid1. =:^( speaking of which... Wouldn't the developers consider to rename raid1 to something more correct? E.g. replicas2 or dup or whatever. RAID1 has ever had the meaning of mirrored devices and the closest to this in btrfs

Re: [PATCH 00/15] btrfs: Hot spare and Auto replace

Hey. You guys may want to update: https://btrfs.wiki.kernel.org/index.php/Project_ideas#Hot_spare_support Cheers, Chris. smime.p7s Description: S/MIME cryptographic signature

Re: bad extent [5993525264384, 5993525280768), type mismatch with chunk

On Sun, 2015-11-15 at 09:29 +0800, Qu Wenruo wrote: > > > If type is wrong, all the extents inside the chunk should be > > > reported > > > as > > > mismatch type with chunk. > > Isn't that the case? At least there are so many reported extents... > > If you posted all the output Sure, I posted

Re: bad extent [5993525264384, 5993525280768), type mismatch with chunk

I just got the backup disk back, also btrfs, which was made via send/receive... It has the same errors during fsck. The main disk still hasn't found any file (apart from a few, others for which none of my hash sums were stored at all) that doesn't verify. So I guess there's definitely some bug

Re: bad extent [5993525264384, 5993525280768), type mismatch with chunk

On Fri, 2015-11-13 at 07:05 +, Duncan wrote: > 8 TiB disks -- are those the disk-managed SMR "archive" disks I've > read > about on a number of threads? Yes... but... > If so, that hardware is almost certainly the cause, as they're known > to > be problematic on current kernels.  While most

Re: bad extent [5993525264384, 5993525280768), type mismatch with chunk

On Sat, 2015-11-14 at 09:22 +0800, Qu Wenruo wrote: > Manually checked they all. thanks a lot :-) > Strangely, they are all OK... although it's a good news for you. Oh man... you're s mean ;-D > They are all tree blocks and are all in metadata block group. and I guess that's...

Re: bad extent [5993525264384, 5993525280768), type mismatch with chunk

I've uploaded the full output of btrfs check on that device to: http://christoph.anton.mitterer.name/tmp/public/cbec6446-898b-11e5-90a4-502690aa641f.xz there are nearly 600k of these error lines... WTF?! Also, the filesystem still mounts (without any errors to dmesg) Any help would be

bad extent [5993525264384, 5993525280768), type mismatch with chunk

Hey. I get these errors on fsck'ing a btrfs: bad extent [5993525264384, 5993525280768), type mismatch with chunk bad extent [5993525280768, 5993525297152), type mismatch with chunk bad extent [5993525297152, 5993525313536), type mismatch with chunk bad extent [5993529442304, 5993529458688), type

Re: bad extent [5993525264384, 5993525280768), type mismatch with chunk

On Fri, 2015-11-13 at 11:23 +0800, Qu Wenruo wrote: > No, "-t 2" means only dump extent tree, no privacy issues at all. > Since only numeric inode/snapshot number and offset inside file. > Or I'll give you a warning on privacy. > > No file name at all, just try it yourself. I'm preparing it...

Re: bad extent [5993525264384, 5993525280768), type mismatch with chunk

Hey. On Fri, 2015-11-13 at 10:13 +0800, Qu Wenruo wrote: > Like this one, if any extent type doesn't match with its chunk, like > metadata extent in a data chunk, btrfsck will report like that. So these errors... are they anything serious? I.e. like data loss/corruption? Or is it more a

Re: bad extent [5993525264384, 5993525280768), type mismatch with chunk

On Fri, 2015-11-13 at 11:23 +0800, Qu Wenruo wrote: > No, "-t 2" means only dump extent tree, no privacy issues at all. > Since only numeric inode/snapshot number and offset inside file. > Or I'll give you a warning on privacy. Done...

Re: bad extent [5993525264384, 5993525280768), type mismatch with chunk

On Fri, 2015-11-13 at 10:52 +0800, Qu Wenruo wrote: > You can provide the output of "btrfs-debug-tree -t 2 " to help > further debug. > It would be quite big, so it's better to zip it. That would contain all the filenames, right? Hmm that could be problematic because of privacy issues... >

Re: bad extent [5993525264384, 5993525280768), type mismatch with chunk

And I should perhaps mention one more thing: As I've said I have these two 8TiB disks... one which is basically the master with loads of precious data, the other being a backup from the master, regularly created with incremental btrfs send/receive. Everytime I did this (which is every two months

Re: strange "No space left on device"

On Sun, 2015-11-08 at 20:39 +, Duncan wrote: > Wow, yes!  Good catch, Henk! =:^)  Hugo obviously didn't catch it, > and I > wouldn't have either, as the bad size detection behavior is so > unexpected, it just wouldn't occur to me to look! Hmm... all that *may* be more likely an error of

mkfs.btrfs doesn't detect SSD

Hey. I'm creating a filesystem on Samsung Evo 850 Pro on top of a dm- crypt/LUKS container (with TRIM not being passed on, for the usual security reasons): # mkfs.btrfs --label system /dev/mapper/system btrfs-progs v4.2.2 See http://btrfs.wiki.kernel.org for more information. Label: 

Re: mkfs.btrfs doesn't detect SSD

Hmm in fact it seems to be the kernel who wrongly, detects the type: /sys/block/sdb/queue/rotational = 1 or more like the USB/SATA bridge simply reports it wrong. Anyway, is there a way to override? Or will setting /sys/block/sdb/queue/rotational = 0 give the expected behaviour? Thanks, Chris.

strange "No space left on device"

Hey. I just repeatedly did the following twice on a ~8GB USB stick, under Debian sid (ergo kernel 4.2.0-1-amd64, btrfsprogs 4.2.2-1). First, created some GPT on the stick: Number  Start (sector)End (sector)  Size   Code  Name    12048 1048575   511.0 MiB   EF02  BIOS

Re: strange "No space left on device"

On Sat, 2015-11-07 at 23:30 +, Hugo Mills wrote: >    These are all really small. Well enough for booting =) >    I would suggest running mkfs with --mixed for all of these > filesystems and trying again. I thought btrfs would do that automatically:

<    1   2   3   4   >