[PATCH] btrfs-progs: print-tree: Enhance warning on tree block level mismatch and error handling

This patch enhance the tree block level mismatch by the following methods: 1) Merge same warning branches into one We had two branches showing the same message, and their condition is also the same. Merge them 2) Only skip bad slot The old code skipped all the remaining slots, here we

Re: zerofree btrfs support?

Hey. On Wed, 2018-03-14 at 20:38 +0100, David Sterba wrote: > I have a prototype code for that and after the years, seeing the > request > again, I'm not against adding it as long as it's not advertised as a > security feature. I'd expect that anyone in the security area should know that

Re: [PATCH] btrfs: Verify extent allocated by find_free_extent() won't overlap with extents from previous transaction

On 2018年03月15日 05:09, David Sterba wrote: > On Tue, Feb 13, 2018 at 09:13:32AM +0800, Qu Wenruo wrote: >> There are reports in mail list, even with latest mainline kernel, btrfs >> can't survive a power loss. >> >> Unlike journal based filesystem, btrfs doesn't use journal for such >> work. (log

Re: Ongoing Btrfs stability issues

On 03/14/2018 08:27 PM, Austin S. Hemmelgarn wrote: > On 2018-03-14 14:39, Goffredo Baroncelli wrote: >> On 03/14/2018 01:02 PM, Austin S. Hemmelgarn wrote: >> [...] In btrfs, a checksum mismatch creates an -EIO error during the reading. In a conventional filesystem (or a btrfs

Re: [PATCH 1/2] btrfs: Remove extra run_delayed_items call

On Tue, Feb 13, 2018 at 04:16:48PM +0200, Nikolay Borisov wrote: > Before commit 581227d0d2b8 ("Btrfs: remove the time check in > btrfs_commit_transaction()") there would be a loop in the transaction > commit path that will go on until all existing writers ended their > transaction handles. This

Re: [PATCH] btrfs: Verify extent allocated by find_free_extent() won't overlap with extents from previous transaction

On Tue, Feb 13, 2018 at 09:13:32AM +0800, Qu Wenruo wrote: > There are reports in mail list, even with latest mainline kernel, btrfs > can't survive a power loss. > > Unlike journal based filesystem, btrfs doesn't use journal for such > work. (log tree is an optimization for fsync, not to keep fs

Re: [PATCH] btrfs: volumes: Remove the meaningless condition of minimal nr_devs when allocating a chunk

On Wed, Jan 31, 2018 at 01:56:15PM +0800, Qu Wenruo wrote: > When checking the minimal nr_devs, there is one dead and meaningless > condition: > > if (ndevs < devs_increment * sub_stripes || ndevs < devs_min) { > > > This condition is meaningless,

Re: [PATCH] btrfs: Document parameters of btrfs_reserve_extent

On Tue, Mar 13, 2018 at 09:56:23PM +0800, Anand Jain wrote: > > > On 03/13/2018 06:22 PM, Nikolay Borisov wrote: > > This function is the entry to the extent allocator and as such has > > quite a number of parameters. Some of those have subtle effects on the > > allocation algorithm. Document

Re: zerofree btrfs support?

On Sat, Mar 10, 2018 at 03:55:25AM +0100, Christoph Anton Mitterer wrote: > Hi. > > Just wondered... was it ever planned (or is there some equivalent) to > get support for btrfs in zerofree? The zerofree was requested, eg. here https://bugzilla.kernel.org/show_bug.cgi?id=69121 and there was a

Re: Ongoing Btrfs stability issues

On 2018-03-14 14:39, Goffredo Baroncelli wrote: On 03/14/2018 01:02 PM, Austin S. Hemmelgarn wrote: [...] In btrfs, a checksum mismatch creates an -EIO error during the reading. In a conventional filesystem (or a btrfs filesystem w/o datasum) there is no checksum, so this problem doesn't

Re: [PATCH] btrfs: Handle error from btrfs_uuid_tree_rem call in _btrfs_ioctl_set_received_subvol

On Mon, Mar 12, 2018 at 02:48:09PM +0200, Nikolay Borisov wrote: > As with every function which deals with modifying the btree > btrfs_uuid_tree_rem can fail for any number of reasons (ie. EIO/ENOMEM). > Handle return error value from this function gracefully by aborting the > transaction. > >

Re: Ongoing Btrfs stability issues

On 03/14/2018 01:02 PM, Austin S. Hemmelgarn wrote: [...] >> >> In btrfs, a checksum mismatch creates an -EIO error during the reading. In a >> conventional filesystem (or a btrfs filesystem w/o datasum) there is no >> checksum, so this problem doesn't exist. >> >> I am curious how ZFS solves

Re: [PATCH v2] btrfs: Use sizeof directly instead of a constant variable

On Tue, Mar 13, 2018 at 10:26:06AM +0200, Nikolay Borisov wrote: > The kernel would like to have all stack VLA usage removed[1]. > Unfortunately using an integer constant variable as the size of an > array is still considered a VLA. Instead let's use directly sizeof(var) > which removes the VLA

Re: [PATCH] Btrfs: drop a VLA in btrfs_check_super_csum()

2018-03-14 16:42 GMT+01:00 David Sterba : > We already have a patch for that from a few days ago, but thanks anyway. > > https://patchwork.kernel.org/patch/10277901/ Oh.. OK Best regards, Salvatore -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the

Re: [RFC PATCH] btrfs: check for SB checksum when scanned

On Tue, Mar 13, 2018 at 11:06:37PM +0800, Anand Jain wrote: > We aren't checking the SB csum when the device scanned, > instead we do that when mounting the device, and if the > csum fails we fail the mount. How if we check the csum > when the device is scanned, I can't see any reason for > why

Re: [PATCH] Btrfs: drop a VLA in btrfs_check_super_csum()

On Tue, Mar 13, 2018 at 08:50:22PM +0100, Salvatore Mesoraca wrote: > Avoid a VLA[1] by using a real constant expression instead of a variable. > The compiler should be able to optimize the original code and avoid using > an actual VLA. Anyway this change is useful because it will avoid a false >

Re: [RFC PATCH] btrfs: check for SB checksum when scanned

On Wed, Mar 14, 2018 at 11:20:41AM +0200, Nikolay Borisov wrote: > > > On 13.03.2018 17:06, Anand Jain wrote: > > We aren't checking the SB csum when the device scanned, > > instead we do that when mounting the device, and if the > > csum fails we fail the mount. How if we check the csum > >

Re: [RFC PATCH] btrfs: check for SB checksum when scanned

On Tue, Mar 13, 2018 at 11:06:37PM +0800, Anand Jain wrote: > We aren't checking the SB csum when the device scanned, > instead we do that when mounting the device, and if the > csum fails we fail the mount. How if we check the csum > when the device is scanned, I can't see any reason for > why

Re: [PATCH] btrfs: add missing initialization in btrfs_check_shared

On Wed, Mar 14, 2018 at 09:03:11AM -0600, Edmund Nadolski wrote: > btrfs_check_shared calls find_parent_nodes in a loop. In each > iteration it passes in a share_check struct to request a check for > a shared extent. The share_check::share_count must be re-initialized > to zero for each iteration

Re: [PATCH] btrfs: add missing initialization in btrfs_check_shared

On 14.03.2018 17:03, Edmund Nadolski wrote: > btrfs_check_shared calls find_parent_nodes in a loop. In each > iteration it passes in a share_check struct to request a check for > a shared extent. The share_check::share_count must be re-initialized > to zero for each iteration in order to avoid

Re: [PATCH] missing initialization in btrfs_check_shared

On Wed, Mar 14, 2018 at 3:03 PM, Edmund Nadolski wrote: > This patch addresses an issue that causes fiemap to falsely > report a shared extent. The test case is as follows: > > # cat do_xfs_io > xfs_io -f -d -c "pwrite -b 16k 0 64k" -c "fiemap -v" /media/scratch/file5 > sync

[PATCH] missing initialization in btrfs_check_shared

This patch addresses an issue that causes fiemap to falsely report a shared extent. The test case is as follows: # cat do_xfs_io xfs_io -f -d -c "pwrite -b 16k 0 64k" -c "fiemap -v" /media/scratch/file5 sync xfs_io -c "fiemap -v" /media/scratch/file5 which gives the resulting output: # .

[PATCH] btrfs: add missing initialization in btrfs_check_shared

btrfs_check_shared calls find_parent_nodes in a loop. In each iteration it passes in a share_check struct to request a check for a shared extent. The share_check::share_count must be re-initialized to zero for each iteration in order to avoid using a stale count value from the previous iteration,

[PATCH 2/2 v2] Btrfs-progs: add fsck test for filesystem with shared prealloc extents

From: Filipe Manana Verify that a filesystem check operation (fsck) does not report the following scenario as an error: An extent is shared between two inodes, as a result of clone/reflink operation, and for one of the inodes, lets call it inode A, the extent is referenced

[PATCH 1/2 v2] Btrfs-progs: check, fix false error reports for shared prealloc extents

From: Filipe Manana Under some cases the filesystem checker reports an error when it finds checksum items for an extent that is referenced by an inode as a prealloc extent. Such cases are not an error when the extent is actually shared (was cloned/reflinked) with other inodes

[PATCH] generic/015: Issue sync after deleting the fillup file

This test fails on btrfs due to the presence of delayed processing of file deletes if the file is smaller than 32mb. Initially commit 97575acd7495b412435d06229a6d94ed9a814ada tried to fix a similar failure by bumping the size of the filesystem. However that change had a knock-on effect in that the

Re: WARN when unmounting a subvolume that is being synced

On Wed, Mar 14, 2018 at 09:46:07AM +0200, Nikolay Borisov wrote: > > > On 14.03.2018 05:10, Tycho Andersen wrote: > > Hi all, > > > > I'm getting the WARN below. I think (?) what I'm doing when I get it > > is that I'm unmounting a subvolume while it's being synced (concurrent > > uses of the

Re: [RFC PATCH] btrfs: check for SB checksum when scanned

On 2018-03-14 05:20, Nikolay Borisov wrote: On 13.03.2018 17:06, Anand Jain wrote: We aren't checking the SB csum when the device scanned, instead we do that when mounting the device, and if the csum fails we fail the mount. How if we check the csum when the device is scanned, I can't see any

Re: Ongoing Btrfs stability issues

On 2018-03-13 15:36, Goffredo Baroncelli wrote: On 03/12/2018 10:48 PM, Christoph Anton Mitterer wrote: On Mon, 2018-03-12 at 22:22 +0100, Goffredo Baroncelli wrote: Unfortunately no, the likelihood might be 100%: there are some patterns which trigger this problem quite easily. See The link

RE: [PATCH v3] kernel.h: Skip single-eval logic on literals in min()/max()

From: Kees Cook > Sent: 13 March 2018 22:15 ... > I'll send a "const_max()" which will refuse to work on > non-constant-values (so it doesn't get accidentally used on variables > that could be exposed to double-evaluation), and will work for stack > array declarations (to avoid the

Re: [PATCH 1/2] Btrfs-progs: check, fix false error reports for shared prealloc extents

On 2018年03月14日 18:33, Filipe Manana wrote: > On Wed, Mar 14, 2018 at 1:32 AM, Qu Wenruo wrote: >> >> >> On 2018年03月14日 02:47, fdman...@kernel.org wrote: >>> From: Filipe Manana >>> >>> Under some cases the filesystem checker reports an error when it

Re: [PATCH 1/2] Btrfs-progs: check, fix false error reports for shared prealloc extents

On Wed, Mar 14, 2018 at 9:19 AM, Nikolay Borisov wrote: > > > On 13.03.2018 20:47, fdman...@kernel.org wrote: >> From: Filipe Manana >> >> Under some cases the filesystem checker reports an error when it finds >> checksum items for an extent that is

Re: [PATCH 1/2] Btrfs-progs: check, fix false error reports for shared prealloc extents

On Wed, Mar 14, 2018 at 8:19 AM, Nikolay Borisov wrote: > > > On 13.03.2018 20:47, fdman...@kernel.org wrote: >> From: Filipe Manana >> >> Under some cases the filesystem checker reports an error when it finds >> checksum items for an extent that is

Re: [PATCH RFC 3/3] fstests: generic: Check the fs after each FUA writes

On 2018年03月14日 18:27, Amir Goldstein wrote: > On Wed, Mar 14, 2018 at 11:02 AM, Qu Wenruo wrote: >> Basic test case which triggers fsstress with dm-log-writes, and then >> check the fs after each FUA writes. >> With needed infrastructure and special handlers for journal based fs.

Re: [PATCH 1/2] Btrfs-progs: check, fix false error reports for shared prealloc extents

On Wed, Mar 14, 2018 at 1:32 AM, Qu Wenruo wrote: > > > On 2018年03月14日 02:47, fdman...@kernel.org wrote: >> From: Filipe Manana >> >> Under some cases the filesystem checker reports an error when it finds >> checksum items for an extent that is

Re: [PATCH RFC 3/3] fstests: generic: Check the fs after each FUA writes

On Wed, Mar 14, 2018 at 11:02 AM, Qu Wenruo wrote: > Basic test case which triggers fsstress with dm-log-writes, and then > check the fs after each FUA writes. > With needed infrastructure and special handlers for journal based fs. > > Signed-off-by: Qu Wenruo > ---

Re: FS unmountable after RAID/LVM problems

On 2018年03月14日 18:06, Dirk Gouders wrote: > Qu Wenruo writes: > >> On 2018年03月14日 17:36, Dirk Gouders wrote: >>> Qu Wenruo writes: >>> On 2018年03月14日 16:53, Dirk Gouders wrote: > Qu Wenruo writes: > >>

Re: [PATCH 2/3] fstests: log-writes: Add support for METADATA flag

On Wed, Mar 14, 2018 at 11:02 AM, Qu Wenruo wrote: > Signed-off-by: Qu Wenruo Reviewed-by: Amir Goldstein > --- > src/log-writes/log-writes.c | 3 ++- > src/log-writes/log-writes.h | 9 + > 2 files changed, 7 insertions(+), 5

Re: [PATCH 1/3] fstests: log-writes: Add support to output human readable flags

On Wed, Mar 14, 2018 at 11:02 AM, Qu Wenruo wrote: > Also change the flag numeric output to hex. > > Signed-off-by: Qu Wenruo Reviewed-by: Amir Goldstein > --- > src/log-writes/log-writes.c | 70 > -

Re: FS unmountable after RAID/LVM problems

On 2018年03月14日 17:36, Dirk Gouders wrote: > Qu Wenruo writes: > >> On 2018年03月14日 16:53, Dirk Gouders wrote: >>> Qu Wenruo writes: >>> On 2018年03月13日 22:49, Dirk Gouders wrote: [snip] >> >> # btrfs inspect dump-tree -b

Re: FS unmountable after RAID/LVM problems

Qu Wenruo writes: > On 2018年03月14日 16:53, Dirk Gouders wrote: >> Qu Wenruo writes: >> >>> On 2018年03月13日 22:49, Dirk Gouders wrote: >>> [snip] > > # btrfs inspect dump-tree -b 848986112 /dev/loop0p1 > # btrfs inspect dump-tree -b

Re: [PATCH v2 2/2] btrfs: drop optimal argument from find_live_mirror()

On 14.03.2018 10:29, Anand Jain wrote: > Drop optimal argument from the function find_live_mirror() > as we can deduce it in the function itself. Also rename > optimal to preferred_mirror. > > Signed-off-by: Anand Jain Reviewed-by: Nikolay Borisov >

Re: [PATCH v2 1/2] btrfs: drop num argument from find_live_mirror()

On 14.03.2018 10:29, Anand Jain wrote: > Obtain the stripes info from the map directly and so no need > to pass it as an argument. > > Signed-off-by: Anand Jain LGTM Reviewed-by: Nikolay Borisov > --- > v1->v2: > Accepts David's comment to rename

Re: [RFC PATCH] btrfs: check for SB checksum when scanned

On 13.03.2018 17:06, Anand Jain wrote: > We aren't checking the SB csum when the device scanned, > instead we do that when mounting the device, and if the > csum fails we fail the mount. How if we check the csum > when the device is scanned, I can't see any reason for > why not? any idea? So

Re: [PATCH 1/2] Btrfs-progs: check, fix false error reports for shared prealloc extents

On 13.03.2018 20:47, fdman...@kernel.org wrote: > From: Filipe Manana > > Under some cases the filesystem checker reports an error when it finds > checksum items for an extent that is referenced by an inode as a prealloc > extent. Such cases are not an error when the extent

Re: FS unmountable after RAID/LVM problems

On 2018年03月14日 16:53, Dirk Gouders wrote: > Qu Wenruo writes: > >> On 2018年03月13日 22:49, Dirk Gouders wrote: >> [snip] # btrfs inspect dump-tree -b 848986112 /dev/loop0p1 # btrfs inspect dump-tree -b 72089600 /dev/loop0p1 >>> >>> OK. >>> >>> (This mail

[PATCH RFC 3/3] fstests: generic: Check the fs after each FUA writes

Basic test case which triggers fsstress with dm-log-writes, and then check the fs after each FUA writes. With needed infrastructure and special handlers for journal based fs. Signed-off-by: Qu Wenruo --- In my test, xfs and btrfs survives while ext4 would report error during fsck.

[PATCH 2/3] fstests: log-writes: Add support for METADATA flag

Signed-off-by: Qu Wenruo --- src/log-writes/log-writes.c | 3 ++- src/log-writes/log-writes.h | 9 + 2 files changed, 7 insertions(+), 5 deletions(-) diff --git a/src/log-writes/log-writes.c b/src/log-writes/log-writes.c index a872429d..5dc22c24 100644 ---

[PATCH 1/3] fstests: log-writes: Add support to output human readable flags

Also change the flag numeric output to hex. Signed-off-by: Qu Wenruo --- src/log-writes/log-writes.c | 70 - 1 file changed, 63 insertions(+), 7 deletions(-) diff --git a/src/log-writes/log-writes.c b/src/log-writes/log-writes.c index

Re: FS unmountable after RAID/LVM problems

Qu Wenruo writes: > On 2018年03月13日 22:49, Dirk Gouders wrote: > [snip] >>> >>> # btrfs inspect dump-tree -b 848986112 /dev/loop0p1 >>> # btrfs inspect dump-tree -b 72089600 /dev/loop0p1 >> >> OK. >> >> (This mail gets a bit long but I don't want to snip probably

[PATCH v2 2/2] btrfs: drop optimal argument from find_live_mirror()

Drop optimal argument from the function find_live_mirror() as we can deduce it in the function itself. Also rename optimal to preferred_mirror. Signed-off-by: Anand Jain --- I thought I have sent v2 to the ML. But now I realize I didn't. v1->v2: Accepts David's comment

[PATCH v2 1/2] btrfs: drop num argument from find_live_mirror()

Obtain the stripes info from the map directly and so no need to pass it as an argument. Signed-off-by: Anand Jain --- v1->v2: Accepts David's comment to rename %num to %num_stripes. fs/btrfs/volumes.c | 16 1 file changed, 12 insertions(+), 4

Re: [PATCH 1/2] Btrfs-progs: check, fix false error reports for shared prealloc extents

On 13.03.2018 20:47, fdman...@kernel.org wrote: > From: Filipe Manana > > Under some cases the filesystem checker reports an error when it finds > checksum items for an extent that is referenced by an inode as a prealloc > extent. Such cases are not an error when the extent

Re: WARN when unmounting a subvolume that is being synced

On 14.03.2018 05:10, Tycho Andersen wrote: > Hi all, > > I'm getting the WARN below. I think (?) what I'm doing when I get it > is that I'm unmounting a subvolume while it's being synced (concurrent > uses of the subvolume, at least, happy to look into it further if the > stack trace is not so

Re: Btrfs broken in massive transfar

Changing crypt layer didn't solve this probrem... > > On Tue, Mar 13, 2018 at 1:25 PM, MASAKI haruka wrote: > > > journal(Kernel log), 7th try (to be readonly): > > > > > > --- > > > 3月 12 16:25:51 lily kernel: BTRFS info (device dm-6): creating UUID tree > > > 3月 12