[RFC][PATCH v3 10/10] btrfs: use common file type conversion

2018-10-26 Thread Phillip Potter
Deduplicate the btrfs file type conversion implementation - file systems that use the same file types as defined by POSIX do not need to define their own versions and can use the common helper functions decared in fs_types.h and implemented in fs_types.c Signed-off-by: Amir Goldstein

Re: Kernel crash related to LZO compression

2018-10-26 Thread Qu Wenruo
On 2018/10/26 下午10:57, Dmitry Katsubo wrote: > On 2018-10-25 20:49, Chris Murphy wrote: >> I would say the first step no matter what if you're using an older >> kernel, is to boot a current Fedora or Arch live or install media, >> mount the Btrfs and try to read the problem files and see if the

[josef-btrfs:kill-mmap-sem-v5 9/10] htmldocs: mm/filemap.c:2882: warning: Function parameter or member 'mkwrite' not described in 'filemap_page_mkwrite_nommapsem'

2018-10-26 Thread kbuild test robot
tree: https://git.kernel.org/pub/scm/linux/kernel/git/josef/btrfs-next.git kill-mmap-sem-v5 head: c7a3e899803d98ae543468f0026500c651528a6c commit: c52634339df10df75f28787f431b937de3c3c591 [9/10] mm: introduce filemap_page_mkwrite_nommapsem reproduce: make htmldocs All warnings (new ones

[PATCH] Btrfs: remove no longer used logged range variables when logging extents

2018-10-26 Thread fdmanana
From: Filipe Manana The logged_start and logged_end variables, at btrfs_log_changed_extents(), were added in commit 8c6c592831a0 ("btrfs: log csums for all modified extents"). However since the recent simplification for fsync, which makes us wait for all ordered extents to complete before

Re: [PATCH] Btrfs: remove no longer used stuff for tracking pending ordered extents

2018-10-26 Thread Liu Bo
On Fri, Oct 26, 2018 at 9:16 AM wrote: > > From: Filipe Manana > > Tracking pending ordered extents per transaction was introduced in commit > 50d9aa99bd35 ("Btrfs: make sure logged extents complete in the current > transaction V3") and later updated in commit 161c3549b45a ("Btrfs: change > how

[PATCH v3] fstests: btrfs verify hardening agaist duplicate fsid

2018-10-26 Thread Anand Jain
We have a known bug in btrfs, that we let the device path be changed after the device has been mounted. So using this loop hole the new copied device would appears as if its mounted immediately after its been copied. So this test case reproduces this issue. For example: Initially..

[PATCH] Btrfs: remove no longer used stuff for tracking pending ordered extents

2018-10-26 Thread fdmanana
From: Filipe Manana Tracking pending ordered extents per transaction was introduced in commit 50d9aa99bd35 ("Btrfs: make sure logged extents complete in the current transaction V3") and later updated in commit 161c3549b45a ("Btrfs: change how we wait for pending ordered extents"). However now

Re: [PATCH v2 rev log added] fstests: btrfs verify hardening agaist duplicate fsid

2018-10-26 Thread Anand Jain
On 10/26/2018 11:52 PM, Nikolay Borisov wrote: On 26.10.18 г. 18:34 ч., Anand Jain wrote: On 10/26/2018 11:02 PM, Nikolay Borisov wrote: On 8.10.18 г. 21:28 ч., Anand Jain wrote: We have a known bug in btrfs, that we let the device path be changed after the device has been mounted.

Re: [PATCH v2 rev log added] fstests: btrfs verify hardening agaist duplicate fsid

2018-10-26 Thread Nikolay Borisov
On 26.10.18 г. 18:34 ч., Anand Jain wrote: > > > On 10/26/2018 11:02 PM, Nikolay Borisov wrote: >> >> >> On 8.10.18 г. 21:28 ч., Anand Jain wrote: >>> We have a known bug in btrfs, that we let the device path be changed >>> after the device has been mounted. So using this loop hole the new

Re: [PATCH v2 rev log added] fstests: btrfs verify hardening agaist duplicate fsid

2018-10-26 Thread Anand Jain
On 10/26/2018 11:02 PM, Nikolay Borisov wrote: On 8.10.18 г. 21:28 ч., Anand Jain wrote: We have a known bug in btrfs, that we let the device path be changed after the device has been mounted. So using this loop hole the new copied device would appears as if its mounted immediately after

Re: [PATCH v2 rev log added] fstests: btrfs verify hardening agaist duplicate fsid

2018-10-26 Thread Nikolay Borisov
On 8.10.18 г. 21:28 ч., Anand Jain wrote: > We have a known bug in btrfs, that we let the device path be changed > after the device has been mounted. So using this loop hole the new > copied device would appears as if its mounted immediately after its > been copied. So this test case reproduces

Re: Kernel crash related to LZO compression

2018-10-26 Thread Dmitry Katsubo
On 2018-10-25 20:49, Chris Murphy wrote: I would say the first step no matter what if you're using an older kernel, is to boot a current Fedora or Arch live or install media, mount the Btrfs and try to read the problem files and see if the problem still happens. I can't even being to estimate

Re: [PATCH] btrfs-progs: add cli to forget one or all scanned devices

2018-10-26 Thread Nikolay Borisov
On 26.10.18 г. 17:27 ч., Anand Jain wrote: > This patch adds cli > btrfs device forget [dev] > to remove the given device structure in the kernel if the device > is unmounted. If no argument is given it shall remove all stale > (device which are not mounted) from the kernel. > >

Re: [PATCH] btrfs: introduce feature to forget a btrfs device

2018-10-26 Thread Nikolay Borisov
On 26.10.18 г. 17:27 ч., Anand Jain wrote: > Support for a new command 'btrfs dev forget [dev]' is proposed here > to undo the effects of 'btrfs dev scan [dev]'. For this purpose > this patch proposes to use ioctl #5 as it was empty. > IOW(BTRFS_IOCTL_MAGIC, 5, ..) > This patch adds new

[PATCH] btrfs-progs: add cli to forget one or all scanned devices

2018-10-26 Thread Anand Jain
This patch adds cli btrfs device forget [dev] to remove the given device structure in the kernel if the device is unmounted. If no argument is given it shall remove all stale (device which are not mounted) from the kernel. Signed-off-by: Anand Jain --- cmds-device.c | 72

[PATCH v11] Add cli and ioctl to forget scanned device(s)

2018-10-26 Thread Anand Jain
v11: btrfs-progs: Bring the code into the else part of if(forget). Use strerror to print the erorr instead of ret. v10: Make btrfs-progs changes more readable. With an effort to keep the known bug [1] as it is.. [1] The cli 'btrfs device scan --all /dev/sdb' which should

[PATCH] btrfs: introduce feature to forget a btrfs device

2018-10-26 Thread Anand Jain
Support for a new command 'btrfs dev forget [dev]' is proposed here to undo the effects of 'btrfs dev scan [dev]'. For this purpose this patch proposes to use ioctl #5 as it was empty. IOW(BTRFS_IOCTL_MAGIC, 5, ..) This patch adds new ioctl BTRFS_IOC_FORGET_DEV which can be sent from the

Re: [PATCH] btrfs-progs: add cli to forget one or all scanned devices

2018-10-26 Thread Anand Jain
On 10/26/2018 08:21 PM, Nikolay Borisov wrote: On 24.10.2018 07:31, Anand Jain wrote: This patch adds cli btrfs device forget [dev] to remove the given device structure in the kernel if the device is unmounted. If no argument is given it shall remove all stale (device which are not

Re: [PATCH] btrfs: Fix error handling in btrfs_cleanup_ordered_extents

2018-10-26 Thread kbuild test robot
-Borisov/btrfs-Fix-error-handling-in-btrfs_cleanup_ordered_extents/20181026-194005 config: x86_64-randconfig-x014-201842 (attached as .config) compiler: gcc-7 (Debian 7.3.0-1) 7.3.0 reproduce: # save the attached .config to linux build tree make ARCH=x86_64 All error/warnings (new

Re: [PATCH 5/5] btrfs: Replace BUG_ON with ASSERT in find_lock_delalloc_range

2018-10-26 Thread Qu Wenruo
On 2018/10/26 下午7:43, Nikolay Borisov wrote: > lock_delalloc_pages should only return 2 values - 0 in case of success > and -EAGAIN if the range of pages to be locked should be shrunk due to > some of gone. Manual inspections confirms that this is > indeed the case since __process_pages_contig

Re: [PATCH 4/5] btrfs: Sink find_lock_delalloc_range's 'max_bytes' argument

2018-10-26 Thread Qu Wenruo
On 2018/10/26 下午7:43, Nikolay Borisov wrote: > All callers of this function pass BTRFS_MAX_EXTENT_SIZE (128M) so let's > reduce the argument count and make that a local variable. No functional > changes. > > Signed-off-by: Nikolay Borisov Reviewed-by: Qu Wenruo Thanks, Qu > --- >

Re: [PATCH 3/5] btrfs: Remove superfluous check form btrfs_remove_chunk

2018-10-26 Thread Qu Wenruo
On 2018/10/26 下午7:43, Nikolay Borisov wrote: > It's unnecessary to check map->stripes[i].dev for NULL given its value > is already set and dereferenced above the the check. No functional changes. > > Signed-off-by: Nikolay Borisov Reviewed-by: Qu Wenruo Thanks, Qu > --- >

Re: [PATCH 1/5] btrfs: Ensure at least 1g is free for balance

2018-10-26 Thread Hans van Kranenburg
On 10/26/18 2:16 PM, Nikolay Borisov wrote: > > (Adding Chris to CC since he is the original author of the code) > > On 26.10.2018 15:09, Hans van Kranenburg wrote: >> On 10/26/18 1:43 PM, Nikolay Borisov wrote: >>> The first part of balance operation is to shrink every constituting >>> device

Re: [PATCH 2/5] btrfs: Refactor btrfs_can_relocate

2018-10-26 Thread Qu Wenruo
On 2018/10/26 下午7:43, Nikolay Borisov wrote: > btrfs_can_relocate returns 0 when it concludes the given chunk can be > relocated and -1 otherwise. Since this function is used as a predicated > and it return a binary value it makes no sense to have it's return > value as an int so change it to

Re: [PATCH 1/5] btrfs: Ensure at least 1g is free for balance

2018-10-26 Thread Qu Wenruo
On 2018/10/26 下午8:08, Nikolay Borisov wrote: > > > On 26.10.2018 15:04, Qu Wenruo wrote: >> >> >> On 2018/10/26 下午7:43, Nikolay Borisov wrote: >>> The first part of balance operation is to shrink every constituting >>> device to ensure there is free space for chunk allocation. However, the

Re: [PATCH] btrfs: Fix error handling in btrfs_cleanup_ordered_extents

2018-10-26 Thread kbuild test robot
-Borisov/btrfs-Fix-error-handling-in-btrfs_cleanup_ordered_extents/20181026-194005 config: i386-randconfig-x007-201842 (attached as .config) compiler: gcc-7 (Debian 7.3.0-1) 7.3.0 reproduce: # save the attached .config to linux build tree make ARCH=i386 All errors (new ones prefixed

Re: [PATCH] btrfs-progs: add cli to forget one or all scanned devices

2018-10-26 Thread Nikolay Borisov
On 24.10.2018 07:31, Anand Jain wrote: > This patch adds cli > btrfs device forget [dev] > to remove the given device structure in the kernel if the device > is unmounted. If no argument is given it shall remove all stale > (device which are not mounted) from the kernel. > > Signed-off-by:

Re: [PATCH 1/5] btrfs: Ensure at least 1g is free for balance

2018-10-26 Thread Nikolay Borisov
(Adding Chris to CC since he is the original author of the code) On 26.10.2018 15:09, Hans van Kranenburg wrote: > On 10/26/18 1:43 PM, Nikolay Borisov wrote: >> The first part of balance operation is to shrink every constituting >> device to ensure there is free space for chunk allocation. >

Re: [PATCH 1/5] btrfs: Ensure at least 1g is free for balance

2018-10-26 Thread Hans van Kranenburg
On 10/26/18 1:43 PM, Nikolay Borisov wrote: > The first part of balance operation is to shrink every constituting > device to ensure there is free space for chunk allocation. A very useful thing to be able to do if there's no unallocated raw disk space left, is use balance to rewrite some

Re: [PATCH 1/5] btrfs: Ensure at least 1g is free for balance

2018-10-26 Thread Nikolay Borisov
On 26.10.2018 15:04, Qu Wenruo wrote: > > > On 2018/10/26 下午7:43, Nikolay Borisov wrote: >> The first part of balance operation is to shrink every constituting >> device to ensure there is free space for chunk allocation. However, the code >> has been buggy ever since its introduction since

Re: [PATCH 1/5] btrfs: Ensure at least 1g is free for balance

2018-10-26 Thread Qu Wenruo
On 2018/10/26 下午7:43, Nikolay Borisov wrote: > The first part of balance operation is to shrink every constituting > device to ensure there is free space for chunk allocation. However, the code > has been buggy ever since its introduction since calculating the space to > shrink > the device by

Re: [PATCH v2] btrfs: Fix error handling in btrfs_cleanup_ordered_extents

2018-10-26 Thread Nikolay Borisov
On 26.10.2018 14:53, Qu Wenruo wrote: > > > On 2018/10/26 下午7:41, Nikolay Borisov wrote: >> Running btrfs/124 in a loop hung up on me sporadically with the >> following call trace: >> btrfs D0 5760 5324 0x >> Call Trace: >> ? __schedule+0x243/0x800 >>

Re: [PATCH v2] btrfs: Fix error handling in btrfs_cleanup_ordered_extents

2018-10-26 Thread Qu Wenruo
On 2018/10/26 下午7:41, Nikolay Borisov wrote: > Running btrfs/124 in a loop hung up on me sporadically with the > following call trace: > btrfs D0 5760 5324 0x > Call Trace: >? __schedule+0x243/0x800 >schedule+0x33/0x90 >

[PATCH 3/5] btrfs: Remove superfluous check form btrfs_remove_chunk

2018-10-26 Thread Nikolay Borisov
It's unnecessary to check map->stripes[i].dev for NULL given its value is already set and dereferenced above the the check. No functional changes. Signed-off-by: Nikolay Borisov --- fs/btrfs/volumes.c | 12 +--- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git

[PATCH 5/5] btrfs: Replace BUG_ON with ASSERT in find_lock_delalloc_range

2018-10-26 Thread Nikolay Borisov
lock_delalloc_pages should only return 2 values - 0 in case of success and -EAGAIN if the range of pages to be locked should be shrunk due to some of gone. Manual inspections confirms that this is indeed the case since __process_pages_contig is where lock_delalloc_pages gets its return value. The

[PATCH 1/5] btrfs: Ensure at least 1g is free for balance

2018-10-26 Thread Nikolay Borisov
The first part of balance operation is to shrink every constituting device to ensure there is free space for chunk allocation. However, the code has been buggy ever since its introduction since calculating the space to shrink the device by was bounded by 1 mb. Most likely the original intention

[PATCH 0/5] Misc cleanups in balance code

2018-10-26 Thread Nikolay Borisov
While investigating the balance hang I came across various inconsistencies in the source. This series aims to fix those. The first patch is (I believe) a fix to a longstanding bug that could cause balance to fail due to ENOSPC. The code no properly ensures that there is at least 1g of

[PATCH 2/5] btrfs: Refactor btrfs_can_relocate

2018-10-26 Thread Nikolay Borisov
btrfs_can_relocate returns 0 when it concludes the given chunk can be relocated and -1 otherwise. Since this function is used as a predicated and it return a binary value it makes no sense to have it's return value as an int so change it to bool. Furthermore, remove a stale leftover comment from

[PATCH 4/5] btrfs: Sink find_lock_delalloc_range's 'max_bytes' argument

2018-10-26 Thread Nikolay Borisov
All callers of this function pass BTRFS_MAX_EXTENT_SIZE (128M) so let's reduce the argument count and make that a local variable. No functional changes. Signed-off-by: Nikolay Borisov --- fs/btrfs/extent_io.c | 11 +-- fs/btrfs/extent_io.h | 2 +-

[PATCH v2] btrfs: Fix error handling in btrfs_cleanup_ordered_extents

2018-10-26 Thread Nikolay Borisov
Running btrfs/124 in a loop hung up on me sporadically with the following call trace: btrfs D0 5760 5324 0x Call Trace: ? __schedule+0x243/0x800 schedule+0x33/0x90 btrfs_start_ordered_extent+0x10c/0x1b0 [btrfs] ?

Re: [PATCH] btrfs: Fix error handling in btrfs_cleanup_ordered_extents

2018-10-26 Thread Holger Hoffstätte
On 10/26/18 13:13, Nikolay Borisov wrote: + if (page_start >= offset && page_end <= (offset + bytes - 1) { fs/btrfs/inode.c: In function 'btrfs_cleanup_ordered_extents': fs/btrfs/inode.c:140:62: error: expected ')' before '{' token if (page_start >= offset && page_end <= (offset +

[PATCH] btrfs: Fix error handling in btrfs_cleanup_ordered_extents

2018-10-26 Thread Nikolay Borisov
Running btrfs/124 in a loop hung up on me sporadically with the following call trace: btrfs D0 5760 5324 0x Call Trace: ? __schedule+0x243/0x800 schedule+0x33/0x90 btrfs_start_ordered_extent+0x10c/0x1b0 [btrfs] ?