[PATCH v2] btrfs: Skip some btrfs_cross_ref_exist() check in nocow path

2018-05-16 Thread Ethan Lien
In nocow path, we check if the extent is snapshotted in btrfs_cross_ref_exist(). We can do the similar check earlier and avoid unnecessary search into extent tree. A fio test on a Intel D-1531, 16GB RAM, SSD RAID-5 machine as follows: [global] group_reporting time_based thread=1 ioengine=libaio b

Re: [PATCH v5 0/3] btrfs: Add three new unprivileged ioctls to allow normal users to call "sub list/show" etc.

2018-05-16 Thread Qu Wenruo
On 2018年05月16日 13:49, Tomohiro Misono wrote: > [based on current misc-next] > > changelog: > v4 -> v5 > - Update error handling of 1st/2nd patch. See each log for details > - Fix misspelling > v3 -> v4 > - call btrfs_next_leaf() after btrfs_search_slot() when the slot > position exceed

Re: [PATCH v5 2/3] btrfs: Add unprivileged ioctl which returns subvolume's ROOT_REF

2018-05-16 Thread Qu Wenruo
On 2018年05月16日 13:49, Tomohiro Misono wrote: > Add unprivileged ioctl BTRFS_IOC_GET_SUBVOL_ROOTREF which > returns ROOT_REF information of the subvolume containing this inode > except the subvolume name (this is because to prevent potential name > leak). The subvolume name will be gained by user

Re: [PATCH v5 1/3] btrfs: Add unprivileged ioctl which returns subvolume information

2018-05-16 Thread Qu Wenruo
On 2018年05月16日 13:49, Tomohiro Misono wrote: > Add new unprivileged ioctl BTRFS_IOC_GET_SUBVOL_INFO which returns > the information of subvolume containing this inode. > (i.e. returns the information in ROOT_ITEM and ROOT_BACKREF.) > > Signed-off-by: Tomohiro Misono > --- > v4 -> v5 > - Up

RE: [PATCH v5 2/3] btrfs: Add unprivileged ioctl which returns subvolume's ROOT_REF

2018-05-16 Thread Gu, Jinxiang
Hi, > -Original Message- > From: linux-btrfs-ow...@vger.kernel.org > [mailto:linux-btrfs-ow...@vger.kernel.org] On Behalf Of Tomohiro Misono > Sent: Wednesday, May 16, 2018 1:50 PM > To: linux-btrfs@vger.kernel.org > Subject: [PATCH v5 2/3] btrfs: Add unprivileged ioctl which returns > s

[PATCH 3/4] btrfs: lzo: Add header length check to avoid slab out of bounds access

2018-05-16 Thread Qu Wenruo
James Harvey reported that some corrupted compressed extent data can lead to various kernel memory corruption. Such corrupted extent data belongs to inode with NODATASUM flags, thus data csum won't help us detecting such bug. If lucky enough, kasan could catch it like: ===

[PATCH 4/4] btrfs: lzo: Hardern inline lzo compressed extent decompression

2018-05-16 Thread Qu Wenruo
Unlike regular lzo compressed extent, inline extent doesn't have Header and only has one Segment. And further more, inlined extent always has csum in its leaf header, it's less possible to have corrupted data. Anyway, still add extra segment header length check. Signed-off-by: Qu Wenruo --- fs/

[PATCH 2/4] btrfs: lzo: Add comment about the how btrfs records its lzo compressed data

2018-05-16 Thread Qu Wenruo
Signed-off-by: Qu Wenruo --- fs/btrfs/lzo.c | 23 +++ 1 file changed, 23 insertions(+) diff --git a/fs/btrfs/lzo.c b/fs/btrfs/lzo.c index 0667ea07f766..3d2ae4c08876 100644 --- a/fs/btrfs/lzo.c +++ b/fs/btrfs/lzo.c @@ -17,6 +17,29 @@ #define LZO_LEN4 +/* + * Btrfs

[PATCH 1/4] btrfs: compression: Add linux/sizes.h for compression.h

2018-05-16 Thread Qu Wenruo
Since compression.h is using SZ_* macros, and if some user only includes compression.h without linux/sizes.h, it will cause compile error. One example is lzo.c, if it uses BTRFS_MAX_COMPRESSED, it would cause compile error. Fix it by adding linux/sizes.h in compression.h Signed-off-by: Qu Wenruo

[PATCH 0/4] btrfs: lzo: Harden decompression callers to avoid

2018-05-16 Thread Qu Wenruo
James Harvey reported pretty strange kernel misbehavior where after reading certain btrfs compressed data, kernel crash with unrelated calltrace. (https://bugzilla.kernel.org/show_bug.cgi?id=199707 and https://www.spinics.net/lists/linux-btrfs/msg77971.html) Thanks for his comprehensive debug hel

RE: [PATCH v5 1/3] btrfs: Add unprivileged ioctl which returns subvolume information

2018-05-16 Thread Gu, Jinxiang
> -Original Message- > From: linux-btrfs-ow...@vger.kernel.org > [mailto:linux-btrfs-ow...@vger.kernel.org] On Behalf Of Tomohiro Misono > Sent: Wednesday, May 16, 2018 1:50 PM > To: linux-btrfs@vger.kernel.org > Subject: [PATCH v5 1/3] btrfs: Add unprivileged ioctl which returns subvolu

[PATCH v14.7 09/14] btrfs: dedupe: Implement btrfs_dedupe_calc_hash interface

2018-05-16 Thread Lu Fengqi
From: Wang Xiaoguang Unlike in-memory or on-disk dedupe method, only SHA256 hash method is supported yet, so implement btrfs_dedupe_calc_hash() interface using SHA256. Signed-off-by: Qu Wenruo Signed-off-by: Wang Xiaoguang Reviewed-by: Josef Bacik Signed-off-by: Lu Fengqi --- v14.7: replace

[PATCH v14.7 04/14] btrfs: dedupe: Introduce function to initialize dedupe info

2018-05-16 Thread Lu Fengqi
From: Wang Xiaoguang Add generic function to initialize dedupe info. Signed-off-by: Qu Wenruo Signed-off-by: Wang Xiaoguang Reviewed-by: Josef Bacik Signed-off-by: Lu Fengqi --- v14.7: fixed the following errors by switching to div_u64. ├── arm-allmodconfig │   └── ERROR:__aeabi_uldivmod-fs-

[PATCH v3 2/2] btrfs: sysfs: Add entry which shows rmdir(2) can work for subvolume

2018-05-16 Thread Misono Tomohiro
Deletion of a subvolume by rmdir(2) has become allowed by the 'commit cd2decf640b1 ("btrfs: Allow rmdir(2) to delete an empty subvolume")'. It is a kind of new feature and this commits add a sysfs entry /sys/fs/btrfs/features/rmdir_subvol to indicate the availability of the feature so that a use

Re: [PATCH 2/2] vfs: dedupe should return EPERM if permission is not granted

2018-05-16 Thread Zygo Blaxell
On Sun, May 13, 2018 at 11:26:39AM -0700, Darrick J. Wong wrote: > On Sun, May 13, 2018 at 06:21:52PM +, Mark Fasheh wrote: > > On Fri, May 11, 2018 at 05:06:34PM -0700, Darrick J. Wong wrote: > > > On Fri, May 11, 2018 at 12:26:51PM -0700, Mark Fasheh wrote: > > > > Right now we return EINVAL

RE: [PATCH v4 3/3] btrfs: Add unprivileged version of ino_lookup ioctl

2018-05-16 Thread Gu, Jinxiang
Hi > -Original Message- > From: linux-btrfs-ow...@vger.kernel.org > [mailto:linux-btrfs-ow...@vger.kernel.org] On Behalf Of Tomohiro Misono > Sent: Friday, May 11, 2018 3:26 PM > To: linux-btrfs@vger.kernel.org > Subject: [PATCH v4 3/3] btrfs: Add unprivileged version of ino_lookup ioctl

Re: [PATCH v2 0/3] btrfs: add read mirror policy

2018-05-16 Thread Anand Jain
On 05/17/2018 06:35 AM, David Sterba wrote: On Wed, May 16, 2018 at 06:03:56PM +0800, Anand Jain wrote: Not yet ready for the integration. As I need to introduce -o no_read_mirror_policy instead of -o read_mirror_policy=- Mount option is mostly likely not the right interface for setting such

Re: [PATCH v2 0/3] btrfs: add read mirror policy

2018-05-16 Thread David Sterba
On Wed, May 16, 2018 at 06:03:56PM +0800, Anand Jain wrote: > Not yet ready for the integration. As I need to introduce > -o no_read_mirror_policy instead of -o read_mirror_policy=- Mount option is mostly likely not the right interface for setting such options, as usual. -- To unsubscribe from thi

[PATCH 2/2] btrfs: tests: drop newline from test_msg strings

2018-05-16 Thread David Sterba
Now that test_err strings do not need the newline, remove them also from the test_msg. Signed-off-by: David Sterba --- fs/btrfs/tests/btrfs-tests.h | 2 +- fs/btrfs/tests/extent-buffer-tests.c | 4 ++-- fs/btrfs/tests/extent-io-tests.c | 8 fs/btrfs/tests/extent-map

[PATCH 1/2] btrfs: tests: add helper for error messages and update them

2018-05-16 Thread David Sterba
The test failures are not clearly visible in the system log as they're printed at INFO level. Add a new helper that is level ERROR. As this touches almost all strings, I took the opportunity to unify them: - decapitalize the first letter as there's a prefix and the text continues after ":" - glu

[PATCH 0/2] Update selfstes strings

2018-05-16 Thread David Sterba
Update test messages so that error reports are visible in the log, few more tweaks of the strings as they're updated. David Sterba (2): btrfs: tests: add helper for error messages and update them btrfs: tests: drop newline from test_msg strings fs/btrfs/tests/btrfs-tests.h | 3 +-

[PATCH 03/18] btrfs-progs: constify pathnames passed as arguments

2018-05-16 Thread jeffm
From: Jeff Mahoney It's unlikely we're going to modify a pathname argument, so codify that and use const. Reviewed-by: Qu Wenruo Signed-off-by: Jeff Mahoney --- chunk-recover.c | 4 ++-- cmds-device.c | 2 +- cmds-fi-usage.c | 6 +++--- cmds-rescue.c | 4 ++-- 4 files changed, 8 insertion

[PATCH v3 00/18] btrfs-progs: qgroups-usability

2018-05-16 Thread jeffm
From: Jeff Mahoney Changes since v2: - Updated Reviewed-by tags were provided. - Fixed the typoed commands that I commented on in the previous posting. - Dropped the btrfs_cleanup_root_info since it's unnecessary with the switch to libbtrfsutil. - Updated the qgroups pathname patch to use libbt

[PATCH 05/18] btrfs-progs: qgroups: add pathname to show output

2018-05-16 Thread jeffm
From: Jeff Mahoney The btrfs qgroup show command currently only exports qgroup IDs, forcing the user to resolve which subvolume each corresponds to. This patch adds pathname resolution to qgroup show so that when the -P option is used, the last column contains the pathname of the root of the sub

[PATCH 14/18] btrfs-progs: pass cmd_struct to clean_args_no_options{,_relaxed}

2018-05-16 Thread jeffm
From: Jeff Mahoney Now that we have a cmd_struct everywhere, we can pass it to clean_args_no_options and have it resolve the usage string from it there. This is necessary for it to pass the cmd_struct to usage() in the next patch. Signed-off-by: Jeff Mahoney --- cmds-balance.c| 6 +++---

[PATCH 01/18] btrfs-progs: quota: Add -W option to rescan to wait without starting rescan

2018-05-16 Thread jeffm
From: Jeff Mahoney This patch adds a new -W option to wait for a rescan without starting a new operation. This is useful for things like xfstests where we want do to do a "btrfs quota enable" and not continue until the subsequent rescan has finished. In addition to documenting the new option in

[PATCH 11/18] btrfs-progs: filesystem balance: split out special handling

2018-05-16 Thread jeffm
From: Jeff Mahoney In preparation to use cmd_struct as the command entry point, we need to split out the 'filesystem balance' handling to not call cmd_balance directly. The reason is that the flags that indicate a command is hidden are a part of cmd_struct and so we can use a cmd_struct as a dir

[PATCH 18/18] btrfs-progs: qgroups: don't print dead qgroups

2018-05-16 Thread jeffm
From: Jeff Mahoney When qgroup items get left behind, we still print them in 'btrfs qgroup show' even though there is nothing to show. Since we now look up the pathname and that means we look up the subvolume, we can filter out first-level qgroups that correspond to roots that have been removed.

[PATCH 15/18] btrfs-progs: pass cmd_struct to usage()

2018-05-16 Thread jeffm
From: Jeff Mahoney Now that every call site has a cmd_struct, we can just pass the cmd_struct to usage to print the usager information. This allows us to interpret the format flags we'll add later in this series to inform the user of which output formats any given command supports. Signed-off-b

[PATCH 13/18] btrfs-progs: pass cmd_struct to command callback function

2018-05-16 Thread jeffm
From: Jeff Mahoney This patch passes the cmd_struct to the command callback function. This has several purposes: It allows the command callback to identify which command was used to call it. It also gives us direct access to the usage associated with that command. Signed-off-by: Jeff Mahoney

[PATCH 04/18] btrfs-progs: btrfs-list: add rb_entry helpers for root_info

2018-05-16 Thread jeffm
From: Jeff Mahoney We use rb_entry all over the place for the root_info pointers. Add a helper to make the code more readable. Signed-off-by: Jeff Mahoney --- btrfs-list.c | 30 -- 1 file changed, 20 insertions(+), 10 deletions(-) diff --git a/btrfs-list.c b/btrfs

[PATCH 08/18] btrfs-progs: subvolume: add quota info to btrfs sub show

2018-05-16 Thread jeffm
From: Jeff Mahoney This patch reports on the first-level qgroup, if any, associated with a particular subvolume. It displays the usage and limit, subject to the usual unit parameters. Signed-off-by: Jeff Mahoney --- cmds-subvolume.c | 51 ++- 1

[PATCH 10/18] btrfs-progs: reorder placement of help declarations for send/receive

2018-05-16 Thread jeffm
From: Jeff Mahoney The usage definitions for send and receive follow the command definitions, which use them. This works because we declare them in commands.h. When we move to using cmd_struct as the entry point, these declarations will be removed, breaking the commands. Since that would be an

[PATCH 12/18] btrfs-progs: use cmd_struct as command entry point

2018-05-16 Thread jeffm
From: Jeff Mahoney Rather than having global command usage and callbacks used to create cmd_structs in the command array, establish the cmd_struct structures separately and use those. The next commit in the series passes the cmd_struct to the command callbacks such that we can access flags and d

[PATCH 17/18] btrfs-progs: handle command groups directly for common case

2018-05-16 Thread jeffm
From: Jeff Mahoney Most command groups just pass their own command group to handle_command_group. We can remove the explicit definitions of command group callbacks by passing the cmd_struct to handle_command_group and allowing it to resolve the group from it. Signed-off-by: Jeff Mahoney --- b

[PATCH 07/18] btrfs-progs: qgroups: introduce btrfs_qgroup_query

2018-05-16 Thread jeffm
From: Jeff Mahoney The only mechanism we have in the progs for searching qgroups is to load all of them and filter the results. This works for qgroup show but to add quota information to 'btrfs subvoluem show' it's pretty wasteful. This patch splits out setting up the search and performing the

[PATCH 06/18] btrfs-progs: qgroups: introduce and use info and limit structures

2018-05-16 Thread jeffm
From: Jeff Mahoney We use structures to pass the info and limit from the kernel as items but store the individual values separately in btrfs_qgroup. We already have a btrfs_qgroup_limit structure that's used for setting the limit. This patch introduces a btrfs_qgroup_info structure and uses tha

[PATCH 16/18] btrfs-progs: add support for output formats

2018-05-16 Thread jeffm
From: Jeff Mahoney This adds a global --format option to request extended output formats from each command. Most of it is plumbing a new cmd_context structure that's established at the beginning of argument parsing into the command callbacks. That structure currently only contains the output mo

[PATCH 09/18] btrfs-progs: help: convert ints used as bools to bool

2018-05-16 Thread jeffm
From: Jeff Mahoney We use an int for 'full', 'all', and 'err' when we really mean a boolean. Reviewed-by: Qu Wenruo Signed-off-by: Jeff Mahoney --- btrfs.c | 14 +++--- help.c | 25 + help.h | 4 ++-- 3 files changed, 22 insertions(+), 21 deletions(-) diff

[PATCH 02/18] btrfs-progs: qgroups: fix misleading index check

2018-05-16 Thread jeffm
From: Jeff Mahoney In print_single_qgroup_table we check the loop index against BTRFS_QGROUP_CHILD, but what we really mean is "last column." Since we have an enum value to indicate the last value, use that instead of assuming that BTRFS_QGROUP_CHILD is always last. Reviewed-by: Qu Wenruo Revi

[PATCH v2 2/5] generic: enable swapfile tests on Btrfs

2018-05-16 Thread Omar Sandoval
From: Omar Sandoval Commit 8c96cfbfe530 ("generic/35[67]: disable swapfile tests on Btrfs") disabled the swapfile tests on Btrfs because it did not support swapfiles at the time. Now that we're adding support, we want these tests to run, but they don't. _require_scratch_swapfile always fails for

[PATCH v2 1/5] xfstests: create swap group

2018-05-16 Thread Omar Sandoval
From: Omar Sandoval I'm going to add a bunch of tests for swap files, so create a group for them and add the existing tests. Signed-off-by: Omar Sandoval --- tests/generic/group | 4 ++-- tests/xfs/group | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/generic/g

[PATCH v2 0/5] xfstests: generic swap file tests

2018-05-16 Thread Omar Sandoval
From: Omar Sandoval Changes since v1: - Add patch 1 to create a group for swap - Add a helper for formatting a swap file instead of open coding everywhere - Use $CHATTR_PROG instead of chattr in a few places that I forgot Thanks! Omar Sandoval (5): xfstests: create swap group generic: en

[PATCH v2 4/5] generic: add test for truncate/fpunch of an active swapfile

2018-05-16 Thread Omar Sandoval
From: Omar Sandoval These should not be allowed. Signed-off-by: Omar Sandoval --- tests/generic/489 | 73 +++ tests/generic/489.out | 8 + tests/generic/group | 1 + 3 files changed, 82 insertions(+) create mode 100755 tests/generic/489 cre

[PATCH v2 5/5] generic: test invalid swap file activation

2018-05-16 Thread Omar Sandoval
From: Omar Sandoval Swap files cannot have holes, and they must at least two pages. swapon(8) and mkswap(8) have stricter restrictions, so add versions of those commands without any restrictions. Signed-off-by: Omar Sandoval --- .gitignore| 2 ++ src/Makefile | 2 +- src

[PATCH v2 3/5] generic: add test for dedupe on an active swapfile

2018-05-16 Thread Omar Sandoval
From: Omar Sandoval Similar to generic/356 that makes sure we can't reflink an active swapfile. Signed-off-by: Omar Sandoval --- tests/generic/488 | 76 +++ tests/generic/488.out | 7 tests/generic/group | 1 + 3 files changed, 84 insertions

Re: [RFC PATCH 5/6] btrfs: add send_stream_version attribute to sysfs

2018-05-16 Thread Omar Sandoval
On Tue, May 08, 2018 at 10:06:50PM -0400, Howard McLauchlan wrote: > From: Filipe David Borba Manana > > So that applications can find out what's the highest send stream > version supported/implemented by the running kernel: > > $ cat /sys/fs/btrfs/send/stream_version > 2 > > [Howard: r

Re: [RFC PATCH 6/6] btrfs: add chattr support for send/receive

2018-05-16 Thread Omar Sandoval
On Tue, May 08, 2018 at 10:06:51PM -0400, Howard McLauchlan wrote: > From: Howard McLauchlan > > Presently btrfs send/receive does not propagate inode attribute flags; > all chattr operations are effectively discarded upon transmission. > > This patch adds kernel support for inode attribute flag

Re: [RFC PATCH 1/6] btrfs: send, bump stream version

2018-05-16 Thread Omar Sandoval
On Tue, May 08, 2018 at 10:06:46PM -0400, Howard McLauchlan wrote: > From: Filipe David Borba Manana > > This increases the send stream version from version 1 to version 2, adding > new commands: > > 1) total data size - used to tell the receiver how much file data the stream >will add or up

Re: [PATCH 14/14] mm: turn on vm_fault_t type checking

2018-05-16 Thread Christoph Hellwig
On Wed, May 16, 2018 at 08:08:29AM -0700, Darrick J. Wong wrote: > Uh, we're changing function signatures /and/ redefinining vm_fault_t? > All in the same 90K patch? > > I /was/ expecting a series of "convert X and all callers/users" > patches followed by a trivial one to switch the definition

Re: vm_fault_t conversion, for real

2018-05-16 Thread Christoph Hellwig
On Wed, May 16, 2018 at 06:22:56AM -0700, Matthew Wilcox wrote: > Perhaps you should try being less of an arsehole if you don't want to > get yelled at? I don't mind when you're an arsehole towards me, but I > do mind when you're an arsehole towards newcomers. How are we supposed > to attract and

Re: [PATCH 0/5] Fix delalloc inodes leaking on btrfs unmount

2018-05-16 Thread David Sterba
On Tue, May 01, 2018 at 04:02:34PM +0200, David Sterba wrote: > On Fri, Apr 27, 2018 at 12:21:49PM +0300, Nikolay Borisov wrote: > > After investigating crashes on generic/176 it turned that the culprit in > > fact > > is the random failure induced by generic/019. As it happens, if on unmount > >

Re: [PATCH 2/5] btrfs: Split btrfs_del_delalloc_inode into 2 functions

2018-05-16 Thread David Sterba
On Fri, May 11, 2018 at 11:39:56AM +0300, Nikolay Borisov wrote: > On 11.05.2018 08:44, Anand Jain wrote: > > On 04/27/2018 05:21 PM, Nikolay Borisov wrote: > >> This is in preparation of fixing delalloc inodes leakage on transaction > >> abort. Also export the new function. > >> > >> Signed-off-by

Re: [PATCH v2 2/2] btrfs: sysfs: Add entry which shows rmdir(2) can work for subvolume

2018-05-16 Thread David Sterba
On Wed, May 16, 2018 at 05:09:27PM +0900, Tomohiro Misono wrote: > Deletion of a subvolume by rmdir(2) has become allowed by the > 'commit cd2decf640b1 ("btrfs: Allow rmdir(2) to delete an empty > subvolume")'. > > It is a kind of new feature and this commits add a sysfs entry > /sys/fs/btrfs/st

Re: [PATCH] btrfs: Skip some btrfs_cross_ref_exist() check in nocow path

2018-05-16 Thread David Sterba
On Wed, May 16, 2018 at 04:28:54PM +0800, Ethan Lien wrote: > In nocow path, we check if the extent is snapshotted in > btrfs_cross_ref_exist(). We can do the similar check earlier and avoid > unnecessary search into extent tree. > > Signed-off-by: Ethan Lien > --- > fs/btrfs/inode.c | 7 +++

Re: [PATCH 14/14] mm: turn on vm_fault_t type checking

2018-05-16 Thread Darrick J. Wong
On Wed, May 16, 2018 at 07:43:48AM +0200, Christoph Hellwig wrote: > Switch vm_fault_t to point to an unsigned int with __bіtwise annotations. > This both catches any old ->fault or ->page_mkwrite instance with plain > compiler type checking, as well as finding more intricate problems with > sparse

Re: [PATCH v2 2/3] btrfs: balance: add args info during start and resume

2018-05-16 Thread Adam Borowski
On Wed, May 16, 2018 at 10:57:57AM +0300, Nikolay Borisov wrote: > On 16.05.2018 05:51, Anand Jain wrote: > > Balance args info is an important information to be reviewed for the > > system audit. So this patch adds it to the kernel log. > > > > Example: > > > > -> btrfs bal start -dprofiles='rai

Re: [PATCH V3] test online label ioctl

2018-05-16 Thread Eric Sandeen
On 5/15/18 7:51 PM, Dave Chinner wrote: On Tue, May 15, 2018 at 10:22:37AM -0500, Eric Sandeen wrote: This tests the online label ioctl that btrfs has, which has been recently proposed for XFS. To run, it requires an updated xfs_io with the label command and a filesystem that supports it A sli

Re: [PATCH v2] btrfs: fix BUG trying to resume balance without resume flag

2018-05-16 Thread David Sterba
On Mon, Apr 30, 2018 at 05:48:45PM +0800, Anand Jain wrote: > We set the BTRFS_BALANCE_RESUME flag in the btrfs_recover_balance(), > which is not called during the remount. So when resuming from the > paused balance we hit BUG. > > kernel: kernel BUG at fs/btrfs/volumes.c:3890! > :: > kernel:

Re: [PATCH v2 3/3] btrfs: balance: add kernel log for end or paused

2018-05-16 Thread Austin S. Hemmelgarn
On 2018-05-16 09:23, Anand Jain wrote: On 05/16/2018 07:25 PM, Austin S. Hemmelgarn wrote: On 2018-05-15 22:51, Anand Jain wrote: Add a kernel log when the balance ends, either for cancel or completed or if it is paused. --- v1->v2: Moved from 2/3 to 3/3   fs/btrfs/volumes.c | 7 +++   1

Re: [PATCH] Btrfs: fix the corruption by reading stale btree blocks

2018-05-16 Thread David Sterba
On Wed, May 16, 2018 at 11:00:14AM +0300, Nikolay Borisov wrote: > > > On 15.05.2018 20:37, Liu Bo wrote: > > If a btree block, aka. extent buffer, is not available in the extent > > buffer cache, it'll be read out from the disk instead, i.e. > > > > btrfs_search_slot() > > read_block_for_sear

Re: vm_fault_t conversion, for real

2018-05-16 Thread Matthew Wilcox
On Wed, May 16, 2018 at 03:03:09PM +0200, Christoph Hellwig wrote: > On Wed, May 16, 2018 at 04:23:47AM -0700, Matthew Wilcox wrote: > > On Wed, May 16, 2018 at 07:43:34AM +0200, Christoph Hellwig wrote: > > > this series tries to actually turn vm_fault_t into a type that can be > > > typechecked a

Re: [PATCH v2 3/3] btrfs: balance: add kernel log for end or paused

2018-05-16 Thread Anand Jain
On 05/16/2018 07:25 PM, Austin S. Hemmelgarn wrote: On 2018-05-15 22:51, Anand Jain wrote: Add a kernel log when the balance ends, either for cancel or completed or if it is paused. --- v1->v2: Moved from 2/3 to 3/3   fs/btrfs/volumes.c | 7 +++   1 file changed, 7 insertions(+) diff --gi

Re: [PATCH 10/14] vgem: separate errno from VM_FAULT_* values

2018-05-16 Thread Matthew Wilcox
On Wed, May 16, 2018 at 03:01:59PM +0200, Christoph Hellwig wrote: > On Wed, May 16, 2018 at 11:53:03AM +0200, Daniel Vetter wrote: > > Reviewed-by: Daniel Vetter > > > > Want me to merge this through drm-misc or plan to pick it up yourself? > > For now I just want a honest discussion if people

Re: [PATCH 14/14] mm: turn on vm_fault_t type checking

2018-05-16 Thread Christoph Hellwig
On Wed, May 16, 2018 at 04:28:13AM -0700, Matthew Wilcox wrote: > On Wed, May 16, 2018 at 07:43:48AM +0200, Christoph Hellwig wrote: > > Switch vm_fault_t to point to an unsigned int with __bіtwise annotations. > > This both catches any old ->fault or ->page_mkwrite instance with plain > > compiler

Re: vm_fault_t conversion, for real

2018-05-16 Thread Christoph Hellwig
On Wed, May 16, 2018 at 04:23:47AM -0700, Matthew Wilcox wrote: > On Wed, May 16, 2018 at 07:43:34AM +0200, Christoph Hellwig wrote: > > this series tries to actually turn vm_fault_t into a type that can be > > typechecked and checks the fallout instead of sprinkling random > > annotations without

Re: [PATCH 10/14] vgem: separate errno from VM_FAULT_* values

2018-05-16 Thread Christoph Hellwig
On Wed, May 16, 2018 at 11:53:03AM +0200, Daniel Vetter wrote: > Reviewed-by: Daniel Vetter > > Want me to merge this through drm-misc or plan to pick it up yourself? For now I just want a honest discussion if people really actually want the vm_fault_t change with the whole picture in place. --

Re: [PATCH 14/14] mm: turn on vm_fault_t type checking

2018-05-16 Thread Matthew Wilcox
On Wed, May 16, 2018 at 07:43:48AM +0200, Christoph Hellwig wrote: > Switch vm_fault_t to point to an unsigned int with __bіtwise annotations. > This both catches any old ->fault or ->page_mkwrite instance with plain > compiler type checking, as well as finding more intricate problems with > sparse

Re: [PATCH v2 3/3] btrfs: balance: add kernel log for end or paused

2018-05-16 Thread Austin S. Hemmelgarn
On 2018-05-15 22:51, Anand Jain wrote: Add a kernel log when the balance ends, either for cancel or completed or if it is paused. --- v1->v2: Moved from 2/3 to 3/3 fs/btrfs/volumes.c | 7 +++ 1 file changed, 7 insertions(+) diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index ce68c

Re: vm_fault_t conversion, for real

2018-05-16 Thread Matthew Wilcox
On Wed, May 16, 2018 at 07:43:34AM +0200, Christoph Hellwig wrote: > this series tries to actually turn vm_fault_t into a type that can be > typechecked and checks the fallout instead of sprinkling random > annotations without context. Yes, why should we have small tasks that newcomers can do when

Re: [PATCH 01/14] orangefs: don't return errno values from ->fault

2018-05-16 Thread Matthew Wilcox
On Wed, May 16, 2018 at 07:43:35AM +0200, Christoph Hellwig wrote: > + rc = orangefs_inode_getattr(file->f_mapping->host, 0, 1, STATX_SIZE); > if (rc) { > gossip_err("%s: orangefs_inode_getattr failed, " > "rc:%d:.\n", __func__, rc); > - return

Re: [PATCH 06/14] btrfs: separate errno from VM_FAULT_* values

2018-05-16 Thread David Sterba
On Wed, May 16, 2018 at 07:43:40AM +0200, Christoph Hellwig wrote: > Signed-off-by: Christoph Hellwig Reviewed-by: David Sterba I can add it to the btrfs queue now, unless you need the patch for the rest of the series. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in

Re: [PATCH] Btrfs: fix the corruption by reading stale btree blocks

2018-05-16 Thread Anand Jain
On 05/16/2018 01:37 AM, Liu Bo wrote: If a btree block, aka. extent buffer, is not available in the extent buffer cache, it'll be read out from the disk instead, i.e. btrfs_search_slot() read_block_for_search() # hold parent and its lock, go to read child btrfs_release_path() rea

Re: [PATCH v2 2/3] btrfs: balance: add args info during start and resume

2018-05-16 Thread Anand Jain
On 05/16/2018 03:57 PM, Nikolay Borisov wrote: On 16.05.2018 05:51, Anand Jain wrote: Balance args info is an important information to be reviewed for the system audit. So this patch adds it to the kernel log. Example: -> btrfs bal start -dprofiles='raid1|single',convert=raid5 -mprofiles=

[PATCH] fstests: btrfs/161: test raid1 missing writes

2018-05-16 Thread Anand Jain
Test to make sure that a raid1 device with missed write reads good data when reassembled. Signed-off-by: Anand Jain --- This test case fails as of now. I am sending this to btrfs ML only as it depends on the read_mirror_policy kernel patches which is in the ML. Please use it using the mount op

[PATCH v2 3/3] btrfs: read_mirror_policy ability to reset

2018-05-16 Thread Anand Jain
Adds an ability to change the read_mirror_policy at remount. Signed-off-by: Anand Jain --- fs/btrfs/super.c | 19 --- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c index 7ddecf4178a6..e70592036b95 100644 --- a/fs/btrfs/super.c

[PATCH v2 0/3] btrfs: add read mirror policy

2018-05-16 Thread Anand Jain
Not yet ready for the integration. As I need to introduce -o no_read_mirror_policy instead of -o read_mirror_policy=- to reset the policy as in 3/3. But I am sending this early so that we can use it for btrfs/161 in the ML, and this patch-set is stable enough for the testing. Anand Jain (3): btr

[PATCH v2 2/3] btrfs: add read_mirror_policy parameter devid

2018-05-16 Thread Anand Jain
Adds the mount option: mount -o read_mirror_policy= To set the devid of the device which should be used for read. That means all the normal reads will go to that particular device only. This also helps testing and gives a better control for the test scripts including mount context reads. Signe

[PATCH v2 1/3] btrfs: add mount option read_mirror_policy

2018-05-16 Thread Anand Jain
In case of RAID1 and RAID10 devices are mirror-ed, a read IO can pick any device for reading. This choice of picking a device for reading should be configurable. In short not one policy would satisfy all types of workload and configs. So before we add more policies, this patch-set makes existing $

Re: [PATCH 10/14] vgem: separate errno from VM_FAULT_* values

2018-05-16 Thread Daniel Vetter
On Wed, May 16, 2018 at 07:43:44AM +0200, Christoph Hellwig wrote: > And streamline the code in vgem_fault with early returns so that it is > a little bit more readable. > > Signed-off-by: Christoph Hellwig > --- > drivers/gpu/drm/vgem/vgem_drv.c | 51 +++-- > 1 file

Re: [PATCH] btrfs: Skip some btrfs_cross_ref_exist() check in nocow path

2018-05-16 Thread Nikolay Borisov
On 16.05.2018 11:28, Ethan Lien wrote: > In nocow path, we check if the extent is snapshotted in > btrfs_cross_ref_exist(). We can do the similar check earlier and avoid > unnecessary search into extent tree. > > Signed-off-by: Ethan Lien If this is supposed to be a performance improvement do

[PATCH] btrfs: Skip some btrfs_cross_ref_exist() check in nocow path

2018-05-16 Thread Ethan Lien
In nocow path, we check if the extent is snapshotted in btrfs_cross_ref_exist(). We can do the similar check earlier and avoid unnecessary search into extent tree. Signed-off-by: Ethan Lien --- fs/btrfs/inode.c | 7 +++ 1 file changed, 7 insertions(+) diff --git a/fs/btrfs/inode.c b/fs/btrf

Re: [PATCH] btrfs-progs: remove BTRFS_LIST_LAYOUT_RAW

2018-05-16 Thread Nikolay Borisov
On 15.05.2018 12:17, Gu Jinxiang wrote: > Since > commit 9005b603d723 ("btrfs-progs: use libbtrfsutil for subvol show"), > BTRFS_LIST_LAYOUT_RAW has no usage. > So, remove it. > > Signed-off-by: Gu Jinxiang Reviewed-by: Nikolay Borisov > --- > btrfs-list.c | 20 > btrfs-

[PATCH v2 2/2] btrfs: sysfs: Add entry which shows rmdir(2) can work for subvolume

2018-05-16 Thread Tomohiro Misono
Deletion of a subvolume by rmdir(2) has become allowed by the 'commit cd2decf640b1 ("btrfs: Allow rmdir(2) to delete an empty subvolume")'. It is a kind of new feature and this commits add a sysfs entry /sys/fs/btrfs/static_features/rmdir_subvol to indicate the availability of feature so that a

[PATCH v2 0/2] btrfs: sysfs: Add entry which shows rmdir(2) can work for subvolume

2018-05-16 Thread Tomohiro Misono
[based on current misc-next] - changelog v1 -> v2 - Explicitly set start of enum btrfs_feature_set as 0 in 1st patch - Create new sysfs group which shows static feature (i.e. features which only depends on kernel version). See 2nd path. - This adds new sysfs entry /sys/f

[PATCH v2 1/2] btrfs: sysfs: Use enum/define value intead of magic number

2018-05-16 Thread Tomohiro Misono
Signed-off-by: Tomohiro Misono --- fs/btrfs/sysfs.c | 11 ++- fs/btrfs/sysfs.h | 4 ++-- 2 files changed, 8 insertions(+), 7 deletions(-) diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c index fa6c8c88b250..217d401fe8ae 100644 --- a/fs/btrfs/sysfs.c +++ b/fs/btrfs/sysfs.c @@ -514,10 +51

Re: [PATCH 2/2] btrfs: sysfs: Add entry which shows rmdir(2) can work for subvolume

2018-05-16 Thread Misono Tomohiro
On 2018/05/15 22:22, David Sterba wrote: > On Tue, May 15, 2018 at 04:33:12PM +0900, Misono Tomohiro wrote: >> Deletion of a subvolume by rmdir(2) has become allowed by the >> 'commit cd2decf640b1 ("btrfs: Allow rmdir(2) to delete an empty >> subvolume")'. >> >> It is a kind of new feature and this

Re: [PATCH] Btrfs: fix the corruption by reading stale btree blocks

2018-05-16 Thread Nikolay Borisov
On 15.05.2018 20:37, Liu Bo wrote: > If a btree block, aka. extent buffer, is not available in the extent > buffer cache, it'll be read out from the disk instead, i.e. > > btrfs_search_slot() > read_block_for_search() # hold parent and its lock, go to read child > btrfs_release_path() >

Re: [PATCH v2 2/3] btrfs: balance: add args info during start and resume

2018-05-16 Thread Nikolay Borisov
On 16.05.2018 05:51, Anand Jain wrote: > Balance args info is an important information to be reviewed for the > system audit. So this patch adds it to the kernel log. > > Example: > > -> btrfs bal start -dprofiles='raid1|single',convert=raid5 > -mprofiles='raid1|single',convert=raid5 /btrfs >

Re: [PATCH v2 3/3] btrfs: balance: add kernel log for end or paused

2018-05-16 Thread Nikolay Borisov
On 16.05.2018 05:51, Anand Jain wrote: > Add a kernel log when the balance ends, either for cancel or completed > or if it is paused. Reviewed-by: Nikolay Borisov > --- > v1->v2: Moved from 2/3 to 3/3 > > fs/btrfs/volumes.c | 7 +++ > 1 file changed, 7 insertions(+) > > diff --git a/fs/

Re: [PATCH v2 1/3] btrfs: balance: prefix kernel logs

2018-05-16 Thread Nikolay Borisov
On 16.05.2018 05:51, Anand Jain wrote: > Kernel logs are very important for the forensic investigations of the > issues in general make it easy to use it. This patch adds 'balance:' > prefix so that it can be easily searched. > > Signed-off-by: Anand Jain Straightforward: Reviewed-by: Nikolay

Re: [PATCH 5/6] Btrfs: grab write lock directly if write_lock_level is the max level

2018-05-16 Thread Nikolay Borisov
On 15.05.2018 20:52, Liu Bo wrote: > In case of (cow && (p->keep_locks || p->lowest_level)), write_lock_level > is the max level, and we should grab write lock of root node from the very > beginning. THis needs to be expanded to explain what are the adverse effects (if any) without this commit.

Re: [PATCH 4/6] Btrfs: remove unused check of skip_locking

2018-05-16 Thread Qu Wenruo
On 2018年05月16日 15:03, Nikolay Borisov wrote: > > > On 15.05.2018 20:52, Liu Bo wrote: >> The check is superfluous since all of callers who set search_for_commit >> also have skip_locking set. > > This is false. For example btrfs_qgroup_rescan_worker sets > search_commit_root = 1 but doesn't se

Re: [PATCH 4/6] Btrfs: remove unused check of skip_locking

2018-05-16 Thread Nikolay Borisov
On 15.05.2018 20:52, Liu Bo wrote: > The check is superfluous since all of callers who set search_for_commit > also have skip_locking set. This is false. For example btrfs_qgroup_rescan_worker sets search_commit_root = 1 but doesn't set skip locking. So either your assumption is wrong or the cod