Re: btrfs check --repair: failed to repair damaged filesystem, aborting

2017-05-02 Thread Roman Mamedov
On Tue, 2 May 2017 23:17:11 -0700 Marc MERLIN wrote: > On Tue, May 02, 2017 at 11:00:08PM -0700, Marc MERLIN wrote: > > David, > > > > I think you maintain btrfs-progs, but I'm not sure if you're in charge > > of check --repair. > > Could you comment on the bottom of the mail, namely: > > > fai

Re: btrfs check --repair: failed to repair damaged filesystem, aborting

2017-05-02 Thread Marc MERLIN
On Tue, May 02, 2017 at 11:00:08PM -0700, Marc MERLIN wrote: > David, > > I think you maintain btrfs-progs, but I'm not sure if you're in charge > of check --repair. > Could you comment on the bottom of the mail, namely: > > failed to repair damaged filesystem, aborting > > So, I'm out of luck no

Re: btrfs check --repair: failed to repair damaged filesystem, aborting

2017-05-02 Thread Marc MERLIN
David, I think you maintain btrfs-progs, but I'm not sure if you're in charge of check --repair. Could you comment on the bottom of the mail, namely: > failed to repair damaged filesystem, aborting > So, I'm out of luck now, full wipe and 3-5 day rebuild? Thanks, Marc Rest: On Tue, May 02, 201

Re: [PATCH] Btrfs: fix the mount failure due to missing devices

2017-05-02 Thread Adam Borowski
On Tue, May 02, 2017 at 05:48:40PM -0600, Liu Bo wrote: > Say there is a raid1 btrfs which consists of two disks, after one disk > becomes unavailable, we can still mount it in degraded mode once, for > the second mount it would refuse to mount it with an error > > "BTRFS warning (device sdf): mis

[PATCH] btrfs-progs: check: Fix heap use after free.

2017-05-02 Thread Su Yue
fsck/004-no-dir-index makes valgrinds complaining about Invalid read. ==31890== Invalid read of size 1 ==31890==at 0x453D09: repair_inode_backrefs (cmds-check.c:2690) ==31890==by 0x453D09: check_inode_recs (cmds-check.c:3330) ==31890==by 0x453D09: check_fs_root (cmds-check.c:4012) ==318

[PATCH] Btrfs: fix the mount failure due to missing devices

2017-05-02 Thread Liu Bo
Say there is a raid1 btrfs which consists of two disks, after one disk becomes unavailable, we can still mount it in degraded mode once, for the second mount it would refuse to mount it with an error "BTRFS warning (device sdf): missing devices (1) exceeds the limit (0), writeable mount is not al

Re: Can I see what device was used to mount btrfs?

2017-05-02 Thread Adam Borowski
On Tue, May 02, 2017 at 10:15:06PM +0200, Kai Krakow wrote: > Ideally, the btrfs wouldn't even appear in /dev until it was assembled > by udev. But apparently that's not the case, and I think this is where > the problems come from. I wish, btrfs would not show up as device nodes > in /dev that the

Re: Can I see what device was used to mount btrfs?

2017-05-02 Thread Kai Krakow
Am Tue, 2 May 2017 21:50:19 +0200 schrieb Goffredo Baroncelli : > On 2017-05-02 20:49, Adam Borowski wrote: > >> It could be some daemon that waits for btrfs to become complete. > >> Do we have something? > > Such a daemon would also have to read the chunk tree. > > I don't think that a daemo

Re: 4.11 relocate crash, null pointer + rolling back a filesystem by X hours?

2017-05-02 Thread Kai Krakow
Am Mon, 1 May 2017 22:56:06 -0600 schrieb Chris Murphy : > On Mon, May 1, 2017 at 9:23 PM, Marc MERLIN wrote: > > Hi Chris, > > > > Thanks for the reply, much appreciated. > > > > On Mon, May 01, 2017 at 07:50:22PM -0600, Chris Murphy wrote: > >> What about btfs check (no repair), without and t

Re: 4.11 relocate crash, null pointer + rolling back a filesystem by X hours?

2017-05-02 Thread Kai Krakow
Am Tue, 2 May 2017 05:01:02 + (UTC) schrieb Duncan <1i5t5.dun...@cox.net>: > Of course on-list I'm somewhat known for my arguments propounding the > notion that any filesystem that's too big to be practically > maintained (including time necessary to restore from backups, should > that be nec

Re: Can I see what device was used to mount btrfs?

2017-05-02 Thread Goffredo Baroncelli
On 2017-05-02 20:49, Adam Borowski wrote: >> It could be some daemon that waits for btrfs to become complete. Do we >> have something? > Such a daemon would also have to read the chunk tree. I don't think that a daemon is necessary. As proof of concept, in the past I developed a mount helper [1]

Re: Can I see what device was used to mount btrfs?

2017-05-02 Thread Adam Borowski
On Tue, May 02, 2017 at 05:19:34PM +0300, Andrei Borzenkov wrote: > On Tue, May 2, 2017 at 4:58 PM, Adam Borowski wrote: > > On Sun, Apr 30, 2017 at 08:47:43AM +0300, Andrei Borzenkov wrote: > >> systemd waits for the final device that makes btrfs complete and mounts > >> it using this device name

btrfs check --repair: failed to repair damaged filesystem, aborting

2017-05-02 Thread Marc MERLIN
(cc trimmed) The one in debian/unstable crashed: gargamel:~# btrfs --version btrfs-progs v4.7.3 gargamel:~# btrfs check --repair /dev/mapper/dshelf2 bytenr mismatch, want=2899180224512, have=3981076597540270796 extent-tree.c:2721: alloc_reserved_tree_block: Assertion `ret` failed. btrfs[0x43e418]

Re: [PATCH] Tidy while loop in end_compressed_writeback

2017-05-02 Thread Sahil Kang
I didn't consider it before, but I agree with your point on explicitly handling the return value from find_get_pages_contig. Thanks for looking over it, Sahil Original Message From: David Sterba Sent: May 2, 2017 9:01:34 AM PDT To: Sahil Kang Cc: linux-btrfs@vger.kernel.org

Re: [PATCH] btrfs-progs: tests: add variable quotation to convert-tests

2017-05-02 Thread David Sterba
On Mon, Apr 24, 2017 at 12:30:00AM +0530, Lakshmipathi.G wrote: > Signed-off-by: Lakshmipathi.G Applied, thanks. > - if [[ $TEST_LOG =~ dump ]]; then > + if [[ "$TEST_LOG" =~ dump ]]; then Please note, that the matched string should not be quoted, since b

Re: [PATCH] Btrfs-progs: run_next_block: Return error code when extent buffer read fails

2017-05-02 Thread David Sterba
On Mon, Apr 24, 2017 at 04:57:17PM +0530, Praveen K Pandey wrote: > run_next_block() ignores the return value of read_tree_block() and > always returns success status. This might cause deal_root_from_list() > to loop infinitely when reading metadata block fails. This bug fixes > the issue by extrac

Re: [PATCH 3/3 v2] Make max_size consistent with nr

2017-05-02 Thread David Sterba
On Fri, Apr 28, 2017 at 11:51:21AM +0200, Christophe de Dinechin wrote: > Since we memset tmpl, max_size==0. This does not seem consistent with nr = 1. > In check_extent_refs, we will call: > > set_extent_dirty(root->fs_info->excluded_extents, >rec->start, >

Re: [PATCH 2/3 v2] Prevent attempt to insert extent record with max_size==0

2017-05-02 Thread David Sterba
On Fri, Apr 28, 2017 at 11:50:23AM +0200, Christophe de Dinechin wrote: > When this happens, we will trip a BUG_ON(end < start) in insert_state > because in check_extent_refs, we use this max_size expecting it's not zero: > > set_extent_dirty(root->fs_info->excluded_extents, >

[PATCH] btrfs: always write superblocks synchronously

2017-05-02 Thread Davidlohr Bueso
Commit b685d3d65ac7 "block: treat REQ_FUA and REQ_PREFLUSH as synchronous" removed REQ_SYNC flag from WRITE_FUA implementation. Since REQ_FUA and REQ_FLUSH flags are stripped from submitted IO when the disk doesn't have volatile write cache and thus effectively make the write async. This was seen t

Re: [PATCH 1/3] Disambiguate between cases where add_tree_backref fails

2017-05-02 Thread David Sterba
On Fri, Apr 28, 2017 at 11:12:35AM +0200, Christophe de Dinechin wrote: > See https://bugzilla.redhat.com/show_bug.cgi?id=1435567 for an example > where the message occurs, > > Signed-off-by: Christophe de Dinechin Applied, thanks. I've moved the strings to next line so they don't overflow 80 co

Re: [PATCH] Tidy while loop in end_compressed_writeback

2017-05-02 Thread David Sterba
On Wed, Apr 19, 2017 at 05:43:19AM -0700, Sahil Kang wrote: > This is my first patch so it's a minor code change. > I think removing the early continue from the loop makes the function a > little easier to follow. I think the opposite. In the current code, it's clear that if ret is 0, then there'

Re: [PATCH] Btrfs: remove an unused variable

2017-05-02 Thread David Sterba
On Tue, May 02, 2017 at 12:25:27PM +0300, Dan Carpenter wrote: > "item" is never used. > > Signed-off-by: Dan Carpenter Reviewed-by: David Sterba -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info a

Re: [PATCH 1/1 linux-next] btrfs: kmap() can't fail

2017-05-02 Thread David Sterba
On Tue, Apr 25, 2017 at 08:11:02PM +0200, Fabian Frederick wrote: > Remove NULL test on kmap() > > Signed-off-by: Fabian Frederick Reviewed-by: David Sterba -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More major

Re: [PATCH 6/7] btrfs: Make flush bios explicitely sync

2017-05-02 Thread David Sterba
On Tue, May 02, 2017 at 05:03:50PM +0200, Jan Kara wrote: > Commit b685d3d65ac7 "block: treat REQ_FUA and REQ_PREFLUSH as > synchronous" removed REQ_SYNC flag from WRITE_{FUA|PREFLUSH|...} > definitions. generic_make_request_checks() however strips REQ_FUA and > REQ_PREFLUSH flags from a bio when

Re: [PATCH] btrfs-progs: btrfs-convert: Add larger device support

2017-05-02 Thread David Sterba
On Tue, May 02, 2017 at 05:06:01PM +0200, David Sterba wrote: > On Thu, Apr 27, 2017 at 03:35:34PM +0530, Lakshmipathi.G wrote: > > Bug: Fail to convert 22TB EXT4 to BTRFS > > https://bugzilla.kernel.org/show_bug.cgi?id=194795 > > Thanks for working on the bug, the proposed fix looks good to me ju

Re: [PATCH] btrfs-progs: btrfs-convert: Add larger device support

2017-05-02 Thread Lakshmipathi.G
On Tue, May 02, 2017 at 05:06:01PM +0200, David Sterba wrote: > On Thu, Apr 27, 2017 at 03:35:34PM +0530, Lakshmipathi.G wrote: > > Bug: Fail to convert 22TB EXT4 to BTRFS > > https://bugzilla.kernel.org/show_bug.cgi?id=194795 > > Thanks for working on the bug, the proposed fix looks good to me ju

Re: [PATCH] btrfs-progs: check: Fix heap use after free.

2017-05-02 Thread David Sterba
On Tue, May 02, 2017 at 02:26:19PM +0800, Su Yue wrote: > The repair_inode_backrefs use the backref again after free while iterating > over backrefs. > So let it continue to next step after free can fix it. Please enhance the changelog. I'm missing some explanation that the new code is still corr

Re: [PATCH] btrfs-progs: btrfs-convert: Add larger device support

2017-05-02 Thread David Sterba
On Thu, Apr 27, 2017 at 03:35:34PM +0530, Lakshmipathi.G wrote: > Bug: Fail to convert 22TB EXT4 to BTRFS > https://bugzilla.kernel.org/show_bug.cgi?id=194795 Thanks for working on the bug, the proposed fix looks good to me judging by the use of extended API. This would need a test though. I'm try

[PATCH 0/7] Fix fallout from changes to FUA and PREFLUSH definitions

2017-05-02 Thread Jan Kara
Hello, this series addresses a performance issue caused by commit b685d3d65ac7 "block: treat REQ_FUA and REQ_PREFLUSH as synchronous". We know for certain this problem significanly regresses (over 10%, in some cases up to 100%) ext4 and btrfs for dbench4 and reaim benchmarks. Based on this I have

[PATCH 6/7] btrfs: Make flush bios explicitely sync

2017-05-02 Thread Jan Kara
Commit b685d3d65ac7 "block: treat REQ_FUA and REQ_PREFLUSH as synchronous" removed REQ_SYNC flag from WRITE_{FUA|PREFLUSH|...} definitions. generic_make_request_checks() however strips REQ_FUA and REQ_PREFLUSH flags from a bio when the storage doesn't report volatile write cache and thus write eff

Re: [PATCH v2] fstests: regression test for nocsum buffered read's repair

2017-05-02 Thread Filipe Manana
On Fri, Apr 28, 2017 at 6:26 PM, Liu Bo wrote: > This is to test whether buffered read retry-repair code is able to work in > raid1 case as expected. > > Please note that without checksum, btrfs doesn't know if the data used to > repair is correct, so repair is more of resync which makes sure that

Re: [PATCH v2] fstests: regression test for nocsum dio read's repair

2017-05-02 Thread Filipe Manana
On Fri, Apr 28, 2017 at 6:26 PM, Liu Bo wrote: > Commit 2dabb3248453 ("Btrfs: Direct I/O read: Work on sectorsized blocks") > introduced this regression. It'd cause 'Segmentation fault' error. > > The upstream fix is > Btrfs: fix segment fault when doing dio read > > Signed-off-by: Liu Bo

Re: [PATCH v3] fstests: regression test for btrfs buffered read's repair

2017-05-02 Thread Filipe Manana
On Fri, Apr 28, 2017 at 6:26 PM, Liu Bo wrote: > This case tests whether buffered read can repair the bad copy if we > have a good copy. > > Commit 20a7db8ab3f2 ("btrfs: add dummy callback for readpage_io_failed > and drop checks") introduced the regression. > > The upstream fix is > Btrfs

Re: [PATCH v3] fstests: regression test for btrfs dio read repair

2017-05-02 Thread Filipe Manana
On Fri, Apr 28, 2017 at 6:25 PM, Liu Bo wrote: > This case tests whether dio read can repair the bad copy if we have > a good copy. > > Commit 2dabb3248453 ("Btrfs: Direct I/O read: Work on sectorsized blocks") > introduced the regression. > > The upstream fix is > Btrfs: fix invalid deref

Re: [PATCH] btrfs-progs: Use more restrict check to read out tree root

2017-05-02 Thread David Sterba
On Tue, Apr 25, 2017 at 04:40:16PM +0800, Qu Wenruo wrote: > For fuzzed image bko-156811-bad-parent-ref-qgroup-verify.raw, it cause > qgroup to report -ENOMEM. > > But the fact is, such image is heavy damaged so there is not valid root > item for extent tree. > > Normal extent tree key in root tr

Re: [PATCH] btrfs-progs: Fix fuzz-test for bko-161821.raw.txt

2017-05-02 Thread David Sterba
On Tue, May 02, 2017 at 03:36:09PM +0800, Lu Fengqi wrote: > Fuzzed image bko-161821.raw cause btrfs check to get segmentation fault. > > The function check_owner_ref attempts to access a non-exist quota tree > when dealing with extent_item [4198400 4096] in the corrupted filesystem. > > The func

Re: [PATCH] btrfs-progs: check: Fix memory leak in check_chunks_and_extents

2017-05-02 Thread David Sterba
On Tue, May 02, 2017 at 11:17:25AM +0800, Qu Wenruo wrote: > fsck/003-shift-offsets makes valgrinds complaining about memory leaks. > ==5910== > ==5910== HEAP SUMMARY: > ==5910== in use at exit: 1,112 bytes in 11 blocks > ==5910== total heap usage: 161 allocs, 150 frees, 164,800 bytes allocat

Re: [PATCH] btrfs-progs: tests: fssum, fix memory leak

2017-05-02 Thread David Sterba
On Wed, Apr 26, 2017 at 09:22:41AM +0800, Lu Fengqi wrote: > Free the alloced memory and close dir before exit. > > Signed-off-by: Lu Fengqi Applied, thanks. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More major

Re: Can I see what device was used to mount btrfs?

2017-05-02 Thread Andrei Borzenkov
On Tue, May 2, 2017 at 4:58 PM, Adam Borowski wrote: > On Sun, Apr 30, 2017 at 08:47:43AM +0300, Andrei Borzenkov wrote: >> I'm chasing issue with btrfs mounts under systemd >> (https://github.com/systemd/systemd/issues/5781) - to summarize, systemd >> waits for the final device that makes btrfs c

Re: [PATCH] btrfs-progs: Fix memory leak when 0 sized block group item is found

2017-05-02 Thread David Sterba
On Tue, Apr 25, 2017 at 04:01:17PM +0800, Qu Wenruo wrote: > When a 0 sized block group item is found, set_extent_bits() will not > really set any bits. > While set_state_private() still inserts allocated block group cache into > block group extent_io_tree. > > So at close_ctree() time, we won't f

Re: [PATCH] btrfs-progs: tests: add variable quotation to fsck-tests

2017-05-02 Thread David Sterba
On Mon, Apr 24, 2017 at 06:59:40PM +0530, Lakshmipathi.G wrote: > Signed-off-by: Lakshmipathi.G Applied, thanks. As this was done out of order with other patches, tests/fsck-tests/026-check-inode-link/test.sh was skipped. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" i

Re: Can I see what device was used to mount btrfs?

2017-05-02 Thread Adam Borowski
On Sun, Apr 30, 2017 at 08:47:43AM +0300, Andrei Borzenkov wrote: > I'm chasing issue with btrfs mounts under systemd > (https://github.com/systemd/systemd/issues/5781) - to summarize, systemd > waits for the final device that makes btrfs complete and mounts it using > this device name. Systemd is

Re: BTRFS warning (device sda7): block group 181491728384 has wrong amount of free space

2017-05-02 Thread Duncan
Marat Khalili posted on Tue, 02 May 2017 09:14:12 +0300 as excerpted: > I cannot understand two messages in syslog, could someone please shed > some light? Here they are: > >> Apr 29 08:54:03 container-name kernel: [792742.662375] BTRFS warning >> (device sda7): block group 181491728384 has wrong

[PATCH] Btrfs: remove an unused variable

2017-05-02 Thread Dan Carpenter
"item" is never used. Signed-off-by: Dan Carpenter diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 1e4e4532f381..337aef86dae5 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -5881,7 +5881,6 @@ static int btrfs_real_readdir(struct file *file, struct dir_context *ctx) struc

[PATCH] btrfs-progs: Fix fuzz-test for bko-161821.raw.txt

2017-05-02 Thread Lu Fengqi
Fuzzed image bko-161821.raw cause btrfs check to get segmentation fault. The function check_owner_ref attempts to access a non-exist quota tree when dealing with extent_item [4198400 4096] in the corrupted filesystem. The function btrfs_new_fs_info always allocate memory for fs_info->quota_root r

Re: btrfs won't mount any more

2017-05-02 Thread Marc Haber
On Thu, Apr 13, 2017 at 10:45:09AM +0200, Marc Haber wrote: > I do have a dd copy of the device now. > > $ sudo losetup --find --show ./dropbtr0.btrfs > $ sudo mount -o skip_balance -t btrfs /dev/loop0 /mnt/tempdisk > > does immediately result in: > > Apr 12 22:37:48 fan kernel: [ 124.742104] l