[PATCH v2] btrfs: volumes: Check memory allocation failure for alloc_btrfs_bio()

2019-04-17 Thread Qu Wenruo
[BUG] With kmalloc failure injection for submit_one_bio(), btrfs can crash like: BUG: unable to handle kernel NULL pointer dereference at 0038 #PF error: [WRITE] PGD 0 P4D 0 Oops: 0002 [#1] PREEMPT SMP PTI CPU: 1 PID: 247 Comm: kworker/u8:4 Not tainted 5.1.0-rc5-custom+ #19

Re: [PATCH] btrfs: volumes: Check memory allocation failure for alloc_btrfs_bio()

2019-04-17 Thread Qu Wenruo
On 2019/4/18 下午2:34, Nikolay Borisov wrote: > > > On 18.04.19 г. 9:28 ч., Qu Wenruo wrote: >> [BUG] >> With kmalloc failure injection for submit_one_bio(), btrfs can crash like: >> >> BUG: unable to handle kernel NULL pointer dereference at 0038 >> #PF error: [WRITE] >> PGD 0 P

Re: [PATCH -next] btrfs: Remove set but not used variable 'fs_info'

2019-04-17 Thread Nikolay Borisov
On 18.04.19 г. 9:43 ч., YueHaibing wrote: > Fixes gcc '-Wunused-but-set-variable' warning: > > fs/btrfs/free-space-tree.c: In function 'load_free_space_tree': > fs/btrfs/free-space-tree.c:1535:24: warning: > variable 'fs_info' set but not used [-Wunused-but-set-variable] > > It's not used sin

[PATCH -next] btrfs: Remove set but not used variable 'fs_info'

2019-04-17 Thread YueHaibing
Fixes gcc '-Wunused-but-set-variable' warning: fs/btrfs/free-space-tree.c: In function 'load_free_space_tree': fs/btrfs/free-space-tree.c:1535:24: warning: variable 'fs_info' set but not used [-Wunused-but-set-variable] It's not used since commit 05e9d3137d47 ("btrfs: get fs_info from block grou

Re: [PATCH] btrfs: volumes: Check memory allocation failure for alloc_btrfs_bio()

2019-04-17 Thread Nikolay Borisov
On 18.04.19 г. 9:28 ч., Qu Wenruo wrote: > [BUG] > With kmalloc failure injection for submit_one_bio(), btrfs can crash like: > > BUG: unable to handle kernel NULL pointer dereference at 0038 > #PF error: [WRITE] > PGD 0 P4D 0 > Oops: 0002 [#1] PREEMPT SMP PTI > CPU: 1 PID

[PATCH] btrfs: volumes: Check memory allocation failure for alloc_btrfs_bio()

2019-04-17 Thread Qu Wenruo
[BUG] With kmalloc failure injection for submit_one_bio(), btrfs can crash like: BUG: unable to handle kernel NULL pointer dereference at 0038 #PF error: [WRITE] PGD 0 P4D 0 Oops: 0002 [#1] PREEMPT SMP PTI CPU: 1 PID: 247 Comm: kworker/u8:4 Not tainted 5.1.0-rc5-custom+ #19

Re: Traces during logical_ino ioctl, slowdown, etc (was Re: [PATCH] Btrfs: do not start a transaction during fiemap)

2019-04-17 Thread Zygo Blaxell
On Wed, Apr 17, 2019 at 05:40:55PM +, Filipe Manana wrote: > On Wed, Apr 17, 2019 at 5:50 PM Zygo Blaxell > wrote: > > > > It looks like btrfs_ioctl_logical_to_ino can set: > > > > > > > > path->search_commit_root = 1; > > > > > > > > just before calling iterate_inodes_from_logical, bu

[PATCH 1/8] mm/fs: don't allow writes to immutable files

2019-04-17 Thread Darrick J. Wong
From: Darrick J. Wong The chattr manpage has this to say about immutable files: "A file with the 'i' attribute cannot be modified: it cannot be deleted or renamed, no link can be created to this file, most of the file's metadata can not be modified, and the file can not be opened in write mode."

[PATCH 8/8] ext4: don't allow any modifications to an immutable file

2019-04-17 Thread Darrick J. Wong
From: Darrick J. Wong Don't allow any modifications to a file that's marked immutable, which means that we have to flush all the writable pages to make the readonly and we have to check the setattr/setflags parameters more closely. Signed-off-by: Darrick J. Wong --- fs/ext4/ioctl.c | 46

[PATCH 7/8] btrfs: don't allow any modifications to an immutable file

2019-04-17 Thread Darrick J. Wong
From: Darrick J. Wong Don't allow any modifications to a file that's marked immutable, which means that we have to flush all the writable pages to make the readonly and we have to check the setattr/setflags parameters more closely. Signed-off-by: Darrick J. Wong --- fs/btrfs/ioctl.c | 47 +++

[PATCH 4/8] xfs: refactor setflags to use setattr code directly

2019-04-17 Thread Darrick J. Wong
From: Darrick J. Wong Refactor the SETFLAGS implementation to use the SETXATTR code directly instead of partially constructing a struct fsxattr and calling bits and pieces of the setxattr code. This reduces code size and becomes necessary in the next patch to maintain the behavior of allowing us

[PATCH 5/8] xfs: clean up xfs_merge_ioc_xflags

2019-04-17 Thread Darrick J. Wong
From: Darrick J. Wong Clean up the calling convention since we're editing the fsxattr struct anyway. Signed-off-by: Darrick J. Wong --- fs/xfs/xfs_ioctl.c | 32 ++-- 1 file changed, 14 insertions(+), 18 deletions(-) diff --git a/fs/xfs/xfs_ioctl.c b/fs/xfs/xfs_i

[PATCH 6/8] xfs: don't allow most setxattr to immutable files

2019-04-17 Thread Darrick J. Wong
From: Darrick J. Wong The chattr manpage has this to say about immutable files: "A file with the 'i' attribute cannot be modified: it cannot be deleted or renamed, no link can be created to this file, most of the file's metadata can not be modified, and the file can not be opened in write mode."

[PATCH v2 0/8] vfs: make immutable files actually immutable

2019-04-17 Thread Darrick J. Wong
Hi all, The chattr(1) manpage has this to say about the immutable bit that system administrators can set on files: "A file with the 'i' attribute cannot be modified: it cannot be deleted or renamed, no link can be created to this file, most of the file's metadata can not be modified, and the file

[PATCH 2/8] xfs: unlock inode when xfs_ioctl_setattr_get_trans can't get transaction

2019-04-17 Thread Darrick J. Wong
From: Darrick J. Wong We passed an inode into xfs_ioctl_setattr_get_trans with join_flags indicating which locks are held on that inode. If we can't allocate a transaction then we need to unlock the inode before we bail out. Signed-off-by: Darrick J. Wong --- fs/xfs/xfs_ioctl.c |2 +- 1 f

[PATCH 3/8] xfs: flush page mappings as part of setting immutable

2019-04-17 Thread Darrick J. Wong
From: Darrick J. Wong The chattr manpage has this to say about immutable files: "A file with the 'i' attribute cannot be modified: it cannot be deleted or renamed, no link can be created to this file, most of the file's metadata can not be modified, and the file can not be opened in write mode."

Re: [PATCH v2 4/4] xfs: don't allow most setxattr to immutable files

2019-04-17 Thread Darrick J. Wong
On Tue, Apr 09, 2019 at 11:24:10AM +0300, Amir Goldstein wrote: > On Tue, Apr 9, 2019 at 6:19 AM Darrick J. Wong > wrote: > > > > From: Darrick J. Wong > > > > The chattr manpage has this to say about immutable files: > > > > "A file with the 'i' attribute cannot be modified: it cannot be delete

Traces during logical_ino ioctl, slowdown, etc (was Re: [PATCH] Btrfs: do not start a transaction during fiemap)

2019-04-17 Thread Filipe Manana
On Wed, Apr 17, 2019 at 5:50 PM Zygo Blaxell wrote: > > On Wed, Apr 17, 2019 at 09:22:44AM +, Filipe Manana wrote: > > On Wed, Apr 17, 2019 at 1:08 AM Zygo Blaxell > > wrote: > > > > > > On Mon, Apr 15, 2019 at 09:29:00AM +0100, fdman...@kernel.org wrote: > > > > From: Filipe Manana > > > >

Re: [PATCH] Btrfs: do not start a transaction during fiemap

2019-04-17 Thread Zygo Blaxell
On Wed, Apr 17, 2019 at 09:22:44AM +, Filipe Manana wrote: > On Wed, Apr 17, 2019 at 1:08 AM Zygo Blaxell > wrote: > > > > On Mon, Apr 15, 2019 at 09:29:00AM +0100, fdman...@kernel.org wrote: > > > From: Filipe Manana > > > > > > During fiemap, for regular extents (non inline) we need to chec

[PATCH 2/2 RFC] btrfs: start transaction in btrfs_ioctl_setflags()

2019-04-17 Thread Anand Jain
When inode attribute flags is set through FS_IOC_SETFLAGS ioctl, there is a bit of duplicate codes, the following things happens twice - start/end_transaction, inode_inc_iversion(), current_time update to inode->i_ctime, and btrfs_update_inode(). These are updated both at btrfs_ioctl_setflags() and

[PATCH 1/2 RFC] btrfs: refactor btrfs_set_props to validate externally

2019-04-17 Thread Anand Jain
In preparation to merge multiple transactions when setting the inode attribute flags, move btrfs_set_props()'s validation part to outside. Signed-off-by: Anand Jain --- fs/btrfs/ioctl.c | 3 +++ fs/btrfs/props.c | 23 +-- fs/btrfs/props.h | 1 + fs/btrfs/xattr.c | 5 +

[PATCH 0/2 RFC] fix duplicate code in btrfs_ioctl_setflags

2019-04-17 Thread Anand Jain
There are two reasons why I am marking this as RFC as of now. I don't see the preparatory patches yet, I wonder if this should be based on for-next-20190415? I couldn't reproduce the issue, which theoretically should exist as in the patch 2/2. Some more cleanups related to the inode flags such as

Re: [PATCH] Btrfs: fix race between send and deduplication that lead to failures and crashes

2019-04-17 Thread David Sterba
On Mon, Apr 15, 2019 at 09:30:18AM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > > Send operates on read only trees and expects them to never change while it > is using them. This is part of its initial design, and this expection is > due to two different reasons: > > 1) When it was

Re: [PATCH] Btrfs: remove no longer used function to run delayed refs asynchronously

2019-04-17 Thread David Sterba
On Wed, Apr 17, 2019 at 10:28:47AM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > > It used to be called from only two places (truncate path and releasing a > transaction handle), but commits 28bad2125767c5 ("btrfs: fix truncate > throttling") and db2462a6ad3dc4 ("btrfs: don't run dela

Re: [PATCH v3.1 1/9] btrfs: delayed-ref: Introduce better documented delayed ref structures

2019-04-17 Thread David Sterba
On Tue, Apr 16, 2019 at 08:01:42AM +0800, Qu Wenruo wrote: > > > On 2019/4/16 上午12:50, David Sterba wrote: > > On Sat, Apr 13, 2019 at 07:48:51AM +0800, Qu Wenruo wrote: > >> > >> > >> On 2019/4/12 下午11:46, David Sterba wrote: > >>> On Thu, Apr 04, 2019 at 02:45:29PM +0800, Qu Wenruo wrote: > >>>

Re: Corrupted files support needed

2019-04-17 Thread Qu Wenruo
On 2019/4/17 下午7:16, Daniel Brunner wrote: > Hi again, > > the check went through, > and no errors were reported... > > I tried mounting the fs again, > and suddenly all read errors disappeared! Then there should be something wrong with the disc connection (but why block layer doesn't complain

Re: [PATCH] btrfs: simplify read_extent_buffer_pages a bit

2019-04-17 Thread David Sterba
On Tue, Apr 16, 2019 at 11:16:33AM +0200, Johannes Thumshirn wrote: > > On Thu, Apr 11, 2019 at 06:59:35PM +0200, David Sterba wrote: > > On Thu, Apr 11, 2019 at 06:40:59PM +0200, Johannes Thumshirn wrote: > > > Currently read_extent_buffer_pages() uses a 4 pass algorithm to read an > > > extent b

Re: Corrupted files support needed

2019-04-17 Thread Daniel Brunner
Hi again, the check went through, and no errors were reported... I tried mounting the fs again, and suddenly all read errors disappeared! A full copy is already running, and after that i will nuke it from orbit. Thank you all for the quick answers, I do not completely understand what exactly wa

Re: [PATCH 0/6] cleanup property and extended attribute set

2019-04-17 Thread Anand Jain
On 17/4/19 5:31 PM, David Sterba wrote: On Tue, Apr 16, 2019 at 06:01:58AM +0800, Anand Jain wrote: On 16/4/19 3:01 am, David Sterba wrote: On Fri, Apr 12, 2019 at 04:02:53PM +0800, Anand Jain wrote: In an attempt to stream line the property and extended attribute set here are the few cle

[PATCH] Btrfs: improve performance on fsync of files with multiple hardlinks

2019-04-17 Thread fdmanana
From: Filipe Manana Commit 41bd6067692382 ("Btrfs: fix fsync of files with multiple hard links in new directories") introduced a path that makes fsync fallback to a full transaction commit in order to avoid losing hard links and new ancestors of the fsynced inode. That path is triggered only when

[PATCH] Btrfs: do not start a transaction at iterate_extent_inodes()

2019-04-17 Thread fdmanana
From: Filipe Manana When finding out which inodes have references on a particular extent, done by backref.c:iterate_extent_inodes(), from the BTRFS_IOC_LOGICAL_INO (both v1 and v2) ioctl and from scrub we use the transaction join api to grab a reference on the currently running transaction, since

Re: [PATCH 0/6] cleanup property and extended attribute set

2019-04-17 Thread Anand Jain
On 17/4/19 5:31 PM, David Sterba wrote: On Tue, Apr 16, 2019 at 06:01:58AM +0800, Anand Jain wrote: On 16/4/19 3:01 am, David Sterba wrote: On Fri, Apr 12, 2019 at 04:02:53PM +0800, Anand Jain wrote: In an attempt to stream line the property and extended attribute set here are the few cle

Re: [PATCH] Btrfs: remove no longer used function to run delayed refs asynchronously

2019-04-17 Thread Qu Wenruo
On 2019/4/17 下午5:28, fdman...@kernel.org wrote: > From: Filipe Manana > > It used to be called from only two places (truncate path and releasing a > transaction handle), but commits 28bad2125767c5 ("btrfs: fix truncate > throttling") and db2462a6ad3dc4 ("btrfs: don't run delayed refs in the end

Re: [PATCH] Btrfs: do not start a transaction during fiemap

2019-04-17 Thread David Sterba
On Wed, Apr 17, 2019 at 09:22:44AM +, Filipe Manana wrote: > On Wed, Apr 17, 2019 at 1:08 AM Zygo Blaxell > wrote: > > > > On Mon, Apr 15, 2019 at 09:29:00AM +0100, fdman...@kernel.org wrote: > > > From: Filipe Manana > > > > > > During fiemap, for regular extents (non inline) we need to chec

Re: [PATCH 0/6] cleanup property and extended attribute set

2019-04-17 Thread David Sterba
On Tue, Apr 16, 2019 at 06:01:58AM +0800, Anand Jain wrote: > > > On 16/4/19 3:01 am, David Sterba wrote: > > On Fri, Apr 12, 2019 at 04:02:53PM +0800, Anand Jain wrote: > >> In an attempt to stream line the property and extended attribute set here > >> are the few cleanup patches. > >> > >> 1/

[PATCH] Btrfs: remove no longer used function to run delayed refs asynchronously

2019-04-17 Thread fdmanana
From: Filipe Manana It used to be called from only two places (truncate path and releasing a transaction handle), but commits 28bad2125767c5 ("btrfs: fix truncate throttling") and db2462a6ad3dc4 ("btrfs: don't run delayed refs in the end transaction logic") removed their calls to this function, s

Re: [PATCH] Btrfs: do not start a transaction during fiemap

2019-04-17 Thread Filipe Manana
On Wed, Apr 17, 2019 at 1:08 AM Zygo Blaxell wrote: > > On Mon, Apr 15, 2019 at 09:29:00AM +0100, fdman...@kernel.org wrote: > > From: Filipe Manana > > > > During fiemap, for regular extents (non inline) we need to check if they > > are shared and if they are, set the shared bit. Checking if an