Re: [PATCH V2] Btrfs: find_free_extent: Do not erroneously skip LOOP_CACHING_WAIT state

[Resending in plain text, apologies.] Hi Chandan, Josef, Chris, I am not sure I understand the fix to the problem. It may happen that when updating the device tree, we need to allocate a new chunk via do_chunk_alloc (while we are holding the device tree root node locked). This is a legitimate

Re: [PATCH 1/2 v3] Btrfs: fix regression when running delayed references

Hi Filipe Manana, My understanding of selecting delayed refs to run or merging them is far from complete. Can you please explain what will happen in the following scenario: 1) Ref1 is created, as you explain 2) Somebody calls __btrfs_run_delayed_refs() and runs Ref1, and we end up with an

Re: [PATCH] Btrfs: fix quick exhaustion of the system array in the superblock

Hi Filipe Manana, Can't the call to btrfs_create_pending_block_groups() cause a deadlock, like in http://www.spinics.net/lists/linux-btrfs/msg48744.html? Because this call updates the device tree, and we may be calling do_chunk_alloc() from find_free_extent() when holding a lock on the device

Re: [auto-]defrag, nodatacow - general suggestions?(was: btrfs: poor performance on deleting many large files?)

On Wed, 2015-12-09 at 13:36 +, Duncan wrote: > Answering the BTW first, not to my knowledge, and I'd be > skeptical.  In > general, btrfs is cowed, and that's the focus.  To the extent that > nocow > is necessary for fragmentation/performance reasons, etc, the idea is > to > try to make cow

Re: [PATCH 1/2 v3] Btrfs: fix regression when running delayed references

On Sun, Dec 13, 2015 at 10:51 AM, Alex Lyakas wrote: > Hi Filipe Manana, > > My understanding of selecting delayed refs to run or merging them is > far from complete. Can you please explain what will happen in the > following scenario: > > 1) Ref1 is created, as you

Re: [PATCH] Btrfs: fix quick exhaustion of the system array in the superblock

On Sun, Dec 13, 2015 at 10:29 AM, Alex Lyakas wrote: > Hi Filipe Manana, > > Can't the call to btrfs_create_pending_block_groups() cause a > deadlock, like in > http://www.spinics.net/lists/linux-btrfs/msg48744.html? Because this > call updates the device tree, and we may

Re: [PATCH 1/2 v3] Btrfs: fix regression when running delayed references

Hi Filipe, Thank you for the explanation. On Sun, Dec 13, 2015 at 5:43 PM, Filipe Manana wrote: > On Sun, Dec 13, 2015 at 10:51 AM, Alex Lyakas wrote: >> Hi Filipe Manana, >> >> My understanding of selecting delayed refs to run or merging them is >>

Re: [PATCH] Btrfs: fix quick exhaustion of the system array in the superblock

Thank you, Filipe. Now it is more clear. Fortunately, in my kernel 3.18 I do not have do_chunk_alloc() calling btrfs_create_pending_block_groups(), so I cannot hit this deadlock. But can hit the issue that this call is meant to fix. Thanks, Alex. On Sun, Dec 13, 2015 at 5:45 PM, Filipe Manana

Re: Very various speed of grep operation on btrfs partition

Ok, I am make another experiment. I am buy new HDD and format it with btrfs file system. Also I increased size of grep data and make bash script wich automate testing: #!/bin/bash #For testing on windows machine #grep_path='/cygdrive/e/Sources/inside' #For testing on new HDD

Determine is file a reflink or not

Is there a way to view the CoW structure, e.g. to know is file just a reflink or it was modified? I copied many files from snapshot with --reflink=always I and want to know which files was modified since the copying. Calculating hashsums seems to be too long thing. -- Ivan Sizov -- To

Re: Still not production ready

On Sun, Dec 13, 2015 at 11:35:08PM +0100, Martin Steigerwald wrote: > Hi! > > For me it is still not production ready. Again I ran into: > > btrfs kworker thread uses up 100% of a Sandybridge core for minutes on random > write into big file > https://bugzilla.kernel.org/show_bug.cgi?id=90401

Re: attacking btrfs filesystems via UUID collisions?

On Fri, 2015-12-11 at 16:06 -0700, Chris Murphy wrote: > For anything but a new and empty Btrfs volume What's the influence of the fs being new/empty? > this hypothetical > attack would be a ton easier to do on LVM and mdadm raid because they > have a tiny amount of metadata to spoof compared to

Still not production ready

Hi! For me it is still not production ready. Again I ran into: btrfs kworker thread uses up 100% of a Sandybridge core for minutes on random write into big file https://bugzilla.kernel.org/show_bug.cgi?id=90401 No matter whether SLES 12 uses it as default for root, no matter whether Fujitsu

Kernel lockup, might be helpful log.

I've finally finished deleting all those nasty unreliable Seagate drives from my array. During the process I crashed my server - over, and over, and over. Completely gone - screen blank, controls unresponsive, no network activity (no, I don't have root on btrfs - data only). Most annoying, but

Re: attacking btrfs filesystems via UUID collisions?

On Sat, 2015-12-12 at 02:34 +0100, S.J. wrote: > A bit more about the dd-is-bad-topic: > > IMHO it doesn't matter at all. Yes, fully agree. > a) For this specific problem here, fixing a security problem > automatically > fixes the risk of data corruption because careless cloning+mounting >

Re: bad extent [5993525264384, 5993525280768), type mismatch with chunk

Laurent Bonnaud wrote on 2015/12/11 15:21 +0100: On 04/12/2015 01:47, Qu Wenruo wrote: [run btrfsck] I did that, too with an old btrfsck version (4.0) and it found the following errors. Then I did a btrfsck --repair, and I have been able to complete my "du -s" test. The next step will we

Re: [auto-]defrag, nodatacow - general suggestions?(was: btrfs: poor performance on deleting many large files?)

Two more on these: On Thu, 2015-11-26 at 00:33 +, Hugo Mills wrote: > 3) When I would actually disable datacow for e.g. a subvolume that > > holds VMs or DBs... what are all the implications? > > Obviously no checksumming, but what happens if I snapshot such a > > subvolume or if I

dear developers, can we have notdatacow + checksumming, plz?

(consider that question being asked with that face on: http://goo.gl/LQaOuA) Hey. I've had some discussions on the list these days about not having checksumming with nodatacow (mostly with Hugo and Duncan). They both basically told me it wouldn't be straight possible with CoW, and Duncan thinks

[PATCH] btrfs-progs: Format change for btrfs fi df

The GlobalReserve space in 'btrfs fi df' is always confusing for a lot of users. As it is not a special chunk type like DATA or METADATA, it's in fact a sub-type of METADATA. So change the output to skip GlobalReserve by default, and adding its total to metadata used. And add a new option

Re: btrfs check inconsistency with raid1, part 1

Chris Murphy wrote on 2015/12/13 21:16 -0700: Part 1= What to do about it? This post. Part 2 = How I got here? I'm still working on the write up, so it's not yet posted. Summary: 2 dev (spinning rust) raid1 for data and metadata. kernel 4.2.6, btrfs-progs 4.2.2 btrfs check with devid 1 and

Re: Will "btrfs check --repair" fix the mounting problem?

Chris Murphy wrote on 2015/12/11 11:24 -0700: On Fri, Dec 11, 2015 at 10:50 AM, Ivan Sizov wrote: Btrfs crashes in few seconds after mounting RW. If it's important: the volume was converted from ext4. "ext2_saved" subvolume still presents. dmesg: [ 625.998387] BTRFS

btrfs check inconsistency with raid1, part 1

Part 1= What to do about it? This post. Part 2 = How I got here? I'm still working on the write up, so it's not yet posted. Summary: 2 dev (spinning rust) raid1 for data and metadata. kernel 4.2.6, btrfs-progs 4.2.2 btrfs check with devid 1 and 2 present produces thousands of scary messages,

Re: [PATCH V2] Btrfs: find_free_extent: Do not erroneously skip LOOP_CACHING_WAIT state

On Sunday 13 Dec 2015 12:18:55 Alex Lyakas wrote: > [Resending in plain text, apologies.] > > Hi Chandan, Josef, Chris, > > I am not sure I understand the fix to the problem. > > It may happen that when updating the device tree, we need to allocate a new > chunk via do_chunk_alloc (while we are

Re: Still not production ready

Qu Wenruo posted on Mon, 14 Dec 2015 10:08:16 +0800 as excerpted: > Martin Steigerwald wrote on 2015/12/13 23:35 +0100: >> Hi! >> >> For me it is still not production ready. > > Yes, this is the *FACT* and not everyone has a good reason to deny it. In the above sentence, I /think/ you (Qu)

Re: Kernel lockup, might be helpful log.

Birdsarenice posted on Sun, 13 Dec 2015 22:55:19 + as excerpted: > Meanwhile, I did get lucky: At one crash I happened to be logged in and > was able to hit dmesg seconds before it went completely. So what I have > here is information that looks like it'll help you track down a >

btrfs check inconsistency with raid1, backstory, part 2

(sdb are a raid0, btrfs receive destination, they don't come up in this dmesg at all) sdd are a raid1, btrfs send source All four devices are in USB enclosures on a new Intel NUC that's had ~32 hours burnin with memtest. Both file systems had received many scrubs before with zero errors all

Re: dear developers, can we have notdatacow + checksumming, plz?

On Mon, 14 Dec 2015 03:59:18 PM Christoph Anton Mitterer wrote: > I've had some discussions on the list these days about not having > checksumming with nodatacow (mostly with Hugo and Duncan). > > They both basically told me it wouldn't be straight possible with CoW, > and Duncan thinks it may

Re: btrfs check inconsistency with raid1, part 1

Thanks for the reply. On Sun, Dec 13, 2015 at 10:48 PM, Qu Wenruo wrote: > > > Chris Murphy wrote on 2015/12/13 21:16 -0700: >> btrfs check with devid 1 and 2 present produces thousands of scary >> messages, e.g. >> checksum verify failed on 714189357056 found E4E3BDB6

Re: Still not production ready

Duncan wrote on 2015/12/14 06:21 +: Qu Wenruo posted on Mon, 14 Dec 2015 10:08:16 +0800 as excerpted: Martin Steigerwald wrote on 2015/12/13 23:35 +0100: Hi! For me it is still not production ready. Yes, this is the *FACT* and not everyone has a good reason to deny it. In the above

Re: Kernel lockup, might be helpful log.

I can't help with the call traces. But several (not all) of the hard resetting link messages are hallmark cases where the SCSI command timer default of 30 seconds looks like it's being hit while the drive itself is hung up doing a sector read recovery (multiple attempts). It's worth seeing if