Re: [PULL] Btrfs for 4.13, part 1

2017-06-27 Thread Nikolay Borisov
On 23.06.2017 18:16, David Sterba wrote: > Hi, > > this is the main batch for 4.13. There are some user visible changes, see > below. The core updates improve error handling (mostly related to bios), with > the usual incremental work on the GFP_NOFS (mis)use removal. All patches have > been in f

btrfs-progs: Replace number with a exsiting macro

2017-06-27 Thread Gu Jinxiang
Use macro SZ_4K to replace number 4096 when define macro BTRFS_SUPER_INFO_SIZE. Signed-off-by: Gu Jinxiang --- disk-io.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/disk-io.h b/disk-io.h index dfe4cf0..5c293eb 100644 --- a/disk-io.h +++ b/disk-io.h @@ -24,7 +24,7 @@ #inc

btrfs-progs: Add some check for sectorsize and nodesize

2017-06-27 Thread Gu Jinxiang
Add some check for sectorsize and nodesize, make it consistent with kernel. Signed-off-by: Gu Jinxiang --- convert/main.c | 6 +- fsfeatures.h | 2 +- mkfs/main.c| 8 +--- utils.c| 21 +++-- 4 files changed, 22 insertions(+), 15 deletions(-) diff --git a

[QUESTION] xfstest generic/015 run failed

2017-06-27 Thread Dai Xiang
Hi! I test on my host and find below issue, i want to know how can fix it? root@ubuntu ~/xfstests# ./check generic/015 FSTYP -- btrfs PLATFORM -- Linux/x86_64 ubuntu 4.12.0-rc6 MKFS_OPTIONS -- /dev/sda2 MOUNT_OPTIONS -- /dev/sda2 /fs/scratch generic/015 - output mismatch (see ~/

[PATCH] fstests: btrfs: reorder the arguments of btrfs replace in btrfs/027

2017-06-27 Thread Lu Fengqi
The option parser only accept options before non-option argument. Although David Sterba has submitted commit df8c7225ba00 ("btrfs: reorder arguments so that options come first"), but this case seems to be left out. Signed-off-by: Lu Fengqi --- tests/btrfs/027 | 4 ++-- 1 file changed, 2 insertio

[PATCH v4 5/6] btrfs: Cleanup num_tolerated_disk_barrier_failures

2017-06-27 Thread Qu Wenruo
As we use per-chunk degradable check, now the global num_tolerated_disk_barrier_failures is of no use. So cleanup it. Signed-off-by: Qu Wenruo --- fs/btrfs/ctree.h | 2 -- fs/btrfs/disk-io.c | 54 -- fs/btrfs/disk-io.h | 2 -- fs/btrfs/vol

[PATCH v4 2/6] btrfs: Do chunk level rw degrade check at mount time

2017-06-27 Thread Qu Wenruo
Now use the btrfs_check_rw_degradable() to do mount time degration check. With this patch, now we can mount with the following case: # mkfs.btrfs -f -m raid1 -d single /dev/sdb /dev/sdc # wipefs -a /dev/sdc # mount /dev/sdb /mnt/btrfs -o degraded As the single data chunk is only in sdb, so it'

[PATCH v4 4/6] btrfs: Allow barrier_all_devices to do chunk level device check

2017-06-27 Thread Qu Wenruo
The last user of num_tolerated_disk_barrier_failures is barrier_all_devices(). But it's can be easily changed to new per-chunk degradable check framework. Signed-off-by: Qu Wenruo --- fs/btrfs/disk-io.c | 16 +++- 1 file changed, 3 insertions(+), 13 deletions(-) diff --git a/fs/btrf

[PATCH v4 6/6] btrfs: Enhance missing device kernel message

2017-06-27 Thread Qu Wenruo
For missing device, btrfs will just refuse to mount with almost meaningless kernel message like: BTRFS info (device vdb6): disk space caching is enabled BTRFS info (device vdb6): has skinny extents BTRFS error (device vdb6): failed to read the system array: -5 BTRFS error (device vdb6): open_c

[PATCH v4 1/6] btrfs: Introduce a function to check if all chunks a OK for degraded rw mount

2017-06-27 Thread Qu Wenruo
Introduce a new function, btrfs_check_rw_degradable(), to check if all chunks in btrfs is OK for degraded rw mount. It provides the new basis for accurate btrfs mount/remount and even runtime degraded mount check other than old one-size-fit-all method. Signed-off-by: Qu Wenruo Signed-off-by: Ana

[PATCH v4 0/6] Chunk level degradable check

2017-06-27 Thread Qu Wenruo
The patchset can be fetched from my github repo: https://github.com/adam900710/linux/tree/degradable The patchset is based on David's for-4.13-part1 branch. Btrfs currently uses num_tolerated_disk_barrier_failures to do global check for tolerated missing device. Although the one-size-fit-all sol

[PATCH v4 3/6] btrfs: Do chunk level degradation check for remount

2017-06-27 Thread Qu Wenruo
Just the same for mount time check, use btrfs_check_rw_degradable() to check if we are OK to be remounted rw. Signed-off-by: Qu Wenruo --- fs/btrfs/super.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c index 74e47794e63f..03fa52008404 1

Re: [PATCH] lib/zstd: use div_u64() to let it build on 32-bit

2017-06-27 Thread Nick Terrell
> Please don't top post. Sorry about that. > Which function needs 1KB of stack space? That's quite a lot. FSE_buildCTable_wksp(), FSE_compress_wksp(), and HUF_readDTableX4() required over 1 KB of stack space. > I can see in [1] that there are some on-stack buffers replaced by > pointers to the

Re: generic/015 run failed with btrfs

2017-06-27 Thread Dai Xiang
On Tue, Jun 27, 2017 at 01:09:17PM -0700, Darrick J. Wong wrote: >[cc linux-btrfs, drop linux-xfs cc] Thank you very much, but another question is that i do not know who to send while i directly send to linux-btrfs has been rejected. >On Tue, Jun 27, 2017 at 07:42:16AM +, Dai, XiangX wrote: >>

Re: How to fix errors that check --mode lomem finds, but --mode normal doesn't?

2017-06-27 Thread Marc MERLIN
On Mon, Jun 26, 2017 at 06:46:16PM +0800, Lu Fengqi wrote: > Thanks for the updated information. I'm sorry that the false alert make > you feel nervous. If you can help me find out whether those are real errors that I need to fix (and can't yet since there is no --repair), or whether they are not

[PATCH] Btrfs-progs: man: nodesize must be power of 2 now

2017-06-27 Thread Liu Bo
This updates mkfs.btrfs's man page with the new limitation that nodesize must be a power of 2 as well. Signed-off-by: Liu Bo --- Documentation/mkfs.btrfs.asciidoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Documentation/mkfs.btrfs.asciidoc b/Documentation/mkfs.btrfs

Re: [PATCH 0/5] v3 block subsystem refcounter conversions

2017-06-27 Thread Kees Cook
On Tue, Jun 27, 2017 at 6:26 AM, Jens Axboe wrote: > On 06/27/2017 05:39 AM, Elena Reshetova wrote: >> Changes in v3: >> No changes in patches apart from trivial rebases, but now by >> default refcount_t = atomic_t and uses all atomic standard operations >> unless CONFIG_REFCOUNT_FULL is enabled.

Re: Lock between userspace and btrfs-cleaner on extent_buffer

2017-06-27 Thread Jeff Mahoney
On 6/13/17 9:05 PM, Sargun Dhillon wrote: > On Thu, Jun 8, 2017 at 11:34 AM, Sargun Dhillon wrote: >> I have a deadlock caught in the wild between two processes -- >> btrfs-cleaner, and userspace process (Docker). Here, you can see both >> of the backtraces. btrfs-cleaner is trying to get a lock o

[PATCH] btrfs: backref, properly iterate the missing keys

2017-06-27 Thread Jeff Mahoney
[This should probably be rolled into Patch 8]. We iterate over the indirect tree looking for refs that don't have key associated with them, look them up, and update the ref with the resolved key. The problem is that when we resolve the key, we've changed where the ref would be located in the tre

Re: [PATCH 08/13] btrfs: convert prelimary reference tracking to use rbtrees

2017-06-27 Thread Jeff Mahoney
On 6/27/17 12:49 PM, David Sterba wrote: > On Tue, Jun 20, 2017 at 10:06:48AM -0600, Edmund Nadolski wrote: >> It's been known for a while that the use of multiple lists >> that are periodically merged was an algorithmic problem within >> btrfs. There are several workloads that don't complete in a

Re: [PATCH 08/13] btrfs: convert prelimary reference tracking to use rbtrees

2017-06-27 Thread Jeff Mahoney
On 6/26/17 1:07 PM, Jeff Mahoney wrote: > On 6/20/17 12:06 PM, Edmund Nadolski wrote: >> It's been known for a while that the use of multiple lists >> that are periodically merged was an algorithmic problem within >> btrfs. There are several workloads that don't complete in any >> reasonable amoun

Re: generic/015 run failed with btrfs

2017-06-27 Thread Darrick J. Wong
[cc linux-btrfs, drop linux-xfs cc] On Tue, Jun 27, 2017 at 07:42:16AM +, Dai, XiangX wrote: > Hi! I test on my host and find this issue, i want to know how can fix it? > > root@localhost ~/xfstests# ./check generic/015 > FSTYP -- btrfs ^ please contact the btrf

Re: [PATCH] Btrfs-progs: convert: do not clear header rev

2017-06-27 Thread Liu Bo
On Mon, Jun 26, 2017 at 05:55:02PM -0600, Liu Bo wrote: > So btrfs_set_header_flags() vs btrfs_set_header_flag, the difference is sort > of > similar to "=" vs "|=", when creating and initialising a new extent buffer, > convert uses the former one which clears header_rev by accident. > Here is th

Re: [PATCH v2] btrfs: remove unused sectorsize member

2017-06-27 Thread David Sterba
On Tue, Jun 27, 2017 at 04:25:52PM +0300, Nikolay Borisov wrote: > The sectorsize member of btrfs_block_group_cache is unused. So remove it, this > reduces the number of holes in the struct. > > With patch: > /* size: 856, cachelines: 14, members: 40 */ > /* sum members: 837, holes: 4, sum holes:

Re: [PATCH 2/2] btrfs: Optimise layout of btrfs_block_group_cache

2017-06-27 Thread David Sterba
On Mon, Jun 26, 2017 at 06:30:34PM +0300, Nikolay Borisov wrote: > > > On 26.06.2017 17:42, Nikolay Borisov wrote: > > With this patch applied pahole stats look like: > > > > /* size: 840, cachelines: 14, members: 40 */ > > /* sum members: 833, holes: 1, sum holes: 7 */ > > /* bit holes: 1, sum

Re: [PATCH 00/13] use rbtrees for preliminary backrefs

2017-06-27 Thread David Sterba
On Tue, Jun 20, 2017 at 10:06:40AM -0600, Edmund Nadolski wrote: > This patch series attempts to improve the performance of backref > searches by changing the prelim_refs implementation to use > rbtrees instead of lists. This also aims to reduce the soft > lockup occurences that can result when a

Re: [PATCH 08/13] btrfs: convert prelimary reference tracking to use rbtrees

2017-06-27 Thread David Sterba
On Tue, Jun 20, 2017 at 10:06:48AM -0600, Edmund Nadolski wrote: > It's been known for a while that the use of multiple lists > that are periodically merged was an algorithmic problem within > btrfs. There are several workloads that don't complete in any > reasonable amount of time (e.g. btrfs/130

Re: [PATCH 07/13] btrfs: remove ref_tree implementation from backref.c

2017-06-27 Thread David Sterba
On Tue, Jun 20, 2017 at 10:06:47AM -0600, Edmund Nadolski wrote: > The BACKREF_FOUND_SHARED checking will be addressed in an upcoming > patch. So this reverts the commit afce772e87c36c7f07f230a76d525025aaf09e41 "btrfs: fix check_shared for fiemap ioctl", this should be mentioned in the changelog.

Re: [PATCH 06/13] btrfs: btrfs_check_shared should manage its own transaction

2017-06-27 Thread David Sterba
On Tue, Jun 20, 2017 at 10:06:46AM -0600, Edmund Nadolski wrote: At least some description should be put here, why the transaction is being moved to btrfs_check_shared. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org M

Re: [PATCH v7 16/22] block: convert to errseq_t based writeback error tracking

2017-06-27 Thread Christoph Hellwig
On Mon, Jun 26, 2017 at 10:34:18AM -0400, Jeff Layton wrote: > The bigger question is -- what about more complex filesystems like > ext4? There are a couple of cases where we can return -EIO or -EROFS on > fsync before filemap_write_and_wait_range is ever called. Like this one > for instance: > >

Re: [PATCH 0/5] v3 block subsystem refcounter conversions

2017-06-27 Thread Jens Axboe
On 06/27/2017 05:39 AM, Elena Reshetova wrote: > Changes in v3: > No changes in patches apart from trivial rebases, but now by > default refcount_t = atomic_t and uses all atomic standard operations > unless CONFIG_REFCOUNT_FULL is enabled. This is a compromize for the > systems that are critical o

[PATCH v2] btrfs: remove unused sectorsize member

2017-06-27 Thread Nikolay Borisov
The sectorsize member of btrfs_block_group_cache is unused. So remove it, this reduces the number of holes in the struct. With patch: /* size: 856, cachelines: 14, members: 40 */ /* sum members: 837, holes: 4, sum holes: 19 */ /* bit holes: 1, sum bit holes: 29 bits */ /* last cacheline: 24 bytes

Re: [PATCH] btrfs: Remove false alert when fiemap range is smaller than on-disk extent

2017-06-27 Thread David Sterba
On Thu, Jun 22, 2017 at 10:01:21AM +0800, Qu Wenruo wrote: > Commit 4751832da990 ("btrfs: fiemap: Cache and merge fiemap extent before > submit it to user") introduced a warning to catch unemitted cached > fiemap extent. > > However such warning doesn't take the following case into consideration:

Re: [PATCH] lib/zstd: use div_u64() to let it build on 32-bit

2017-06-27 Thread David Sterba
Please don't top post. On Tue, Jun 27, 2017 at 05:27:51AM +, Nick Terrell wrote: > Adam, I’ve applied the same patch in my tree. I’ll send out the update [1] > once it's reviewed, since I also reduced the stack usage of functions > using over 1 KB of stack space. Which function needs 1KB of s

Re: [PATCH v3.1 0/7] Chunk level degradable check

2017-06-27 Thread David Sterba
On Tue, Jun 27, 2017 at 07:20:06AM -0400, Austin S. Hemmelgarn wrote: > On 2017-06-26 22:49, Qu Wenruo wrote: > > > > > > At 06/27/2017 09:59 AM, Anand Jain wrote: > >> > >> > >> On 06/27/2017 09:05 AM, Qu Wenruo wrote: > >>> > >>> > >>> At 06/27/2017 02:59 AM, David Sterba wrote: > On Thu,

Re: [PATCH v2] btrfs-progs: lowmem check: Fix false alert about file extent interrupt

2017-06-27 Thread David Sterba
On Tue, Jun 27, 2017 at 01:37:47PM +0800, Lu Fengqi wrote: > On Mon, Jun 26, 2017 at 04:55:04PM +0200, David Sterba wrote: > >On Thu, Jun 22, 2017 at 04:12:56PM +0800, Lu Fengqi wrote: > >> As Qu mentioned in this thread > >> (https://www.spinics.net/lists/linux-btrfs/msg64469.html), compression >

[PATCH 3/5] block: convert blkcg_gq.refcnt from atomic_t to refcount_t

2017-06-27 Thread Elena Reshetova
refcount_t type and corresponding API should be used instead of atomic_t when the variable is used as a reference counter. This allows to avoid accidental refcounter overflows that might lead to use-after-free situations. Signed-off-by: Elena Reshetova Signed-off-by: Hans Liljestrand Signed-off-

[PATCH 4/5] block: convert io_context.active_ref from atomic_t to refcount_t

2017-06-27 Thread Elena Reshetova
refcount_t type and corresponding API should be used instead of atomic_t when the variable is used as a reference counter. This allows to avoid accidental refcounter overflows that might lead to use-after-free situations. Signed-off-by: Elena Reshetova Signed-off-by: Hans Liljestrand Signed-off-

[PATCH 0/5] v3 block subsystem refcounter conversions

2017-06-27 Thread Elena Reshetova
Changes in v3: No changes in patches apart from trivial rebases, but now by default refcount_t = atomic_t and uses all atomic standard operations unless CONFIG_REFCOUNT_FULL is enabled. This is a compromize for the systems that are critical on performance and cannot accept even slight delay on the

[PATCH 1/5] block: convert bio.__bi_cnt from atomic_t to refcount_t

2017-06-27 Thread Elena Reshetova
refcount_t type and corresponding API should be used instead of atomic_t when the variable is used as a reference counter. This allows to avoid accidental refcounter overflows that might lead to use-after-free situations. Signed-off-by: Elena Reshetova Signed-off-by: Hans Liljestrand Signed-off-

[PATCH 5/5] block: convert bsg_device.ref_count from atomic_t to refcount_t

2017-06-27 Thread Elena Reshetova
refcount_t type and corresponding API should be used instead of atomic_t when the variable is used as a reference counter. This allows to avoid accidental refcounter overflows that might lead to use-after-free situations. Signed-off-by: Elena Reshetova Signed-off-by: Hans Liljestrand Signed-off-

[PATCH 2/5] block: convert blk_queue_tag.refcnt from atomic_t to refcount_t

2017-06-27 Thread Elena Reshetova
refcount_t type and corresponding API should be used instead of atomic_t when the variable is used as a reference counter. This allows to avoid accidental refcounter overflows that might lead to use-after-free situations. Signed-off-by: Elena Reshetova Signed-off-by: Hans Liljestrand Signed-off-

Re: [PATCH 1/2] btrfs: remove unused sectorsize member

2017-06-27 Thread kbuild test robot
Hi Nikolay, [auto build test ERROR on v4.12-rc6] [also build test ERROR on next-20170627] [cannot apply to btrfs/next] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url: https://github.com/0day-ci/linux/commits/Nikolay-Borisov/btrfs

Re: [PATCH v3.1 0/7] Chunk level degradable check

2017-06-27 Thread Austin S. Hemmelgarn
On 2017-06-26 22:49, Qu Wenruo wrote: At 06/27/2017 09:59 AM, Anand Jain wrote: On 06/27/2017 09:05 AM, Qu Wenruo wrote: At 06/27/2017 02:59 AM, David Sterba wrote: On Thu, Mar 09, 2017 at 09:34:35AM +0800, Qu Wenruo wrote: Btrfs currently uses num_tolerated_disk_barrier_failures to do

Re: [PULL] Btrfs for 4.13, part 1

2017-06-27 Thread Filipe Manana
On Mon, Jun 26, 2017 at 5:58 PM, Chris Mason wrote: > On 06/23/2017 11:16 AM, David Sterba wrote: >> >> Hi, >> >> this is the main batch for 4.13. There are some user visible changes, see >> below. The core updates improve error handling (mostly related to bios), >> with >> the usual incremental w

Re: [PATCH] btrfs: fix validation of XATTR_ITEM dir items

2017-06-27 Thread Filipe Manana
On Wed, Jun 21, 2017 at 4:46 PM, David Sterba wrote: > The XATTR_ITEM is a type of a directory item so we use the common > validator helper. We have to adjust the limits because of potential > data_len (ie. the xattr value), which is otherwise 0 for other directory > items. Did you run fstests?

[PATCH] btrfs-progs: lowmem: check: Output more detailed information about file extent interrupt

2017-06-27 Thread Lu Fengqi
Make lowmem mode output more detailed information about file extent interrupt. Signed-off-by: Lu Fengqi --- Sorry for any inconvenience caused. Please ignore this message(https://www.spinics.net/lists/linux-btrfs/msg66803.html). cmds-check.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deleti

invalid location in dir intem

2017-06-27 Thread Karel Kočí
Hello I have encountered problem with my btrfs partition. I have file that is reported as missing, but I can't get rid of it. btrfs-check reports: checking extents checking free space cache checking fs roots invalid location in dir item 99 invalid location

[PATCH 3/3] btrfs: Be explicit about usage of min()

2017-06-27 Thread Nikolay Borisov
__btrfs_alloc_chunk contains code which boils down to: ndevs = min(ndevs, devs_max) It's conditional upon devs_max not being 0. However, it cannot really be 0 since it's always set to either BTRFS_MAX_DEVS_SYS_CHUNK or BTRFS_MAX_DEVS(fs_info->chunk_root). So eliminate the condition check and

[PATCH 1/3] btrfs: convert while loop to list_for_each_entry

2017-06-27 Thread Nikolay Borisov
No functional changes, just make the loop a bit more readable Signed-off-by: Nikolay Borisov --- fs/btrfs/volumes.c | 11 ++- 1 file changed, 2 insertions(+), 9 deletions(-) diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index adf32f46a73f..9f4426014490 100644 --- a/fs/btrfs/volum

[PATCH 2/3] btrfs: Use explicit round_down call rather than open-coding it

2017-06-27 Thread Nikolay Borisov
No functional changes Signed-off-by: Nikolay Borisov --- fs/btrfs/volumes.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index 9f4426014490..07cfcc507ba6 100644 --- a/fs/btrfs/volumes.c +++ b/fs/btrfs/volumes.c @@ -4752,7 +4752,7 @@