Re: Recover data from damage disk in "array"

2021-01-22 Thread Chris Murphy
On Mon, Jan 18, 2021 at 5:02 PM Hérikz Nawarro wrote: > > Hello everyone, > > I got an array of 4 disks with btrfs configured with data single and > metadata dup, one disk of this array was plugged with a bad sata cable > that broke the plastic part of the data port (the pins still intact), > i st

Re: [PATCH v13 01/42] block: add bio_add_zone_append_page

2021-01-22 Thread Chaitanya Kulkarni
On 1/21/21 10:24 PM, Naohiro Aota wrote: > From: Johannes Thumshirn > > Add bio_add_zone_append_page(), a wrapper around bio_add_hw_page() which > is intended to be used by file systems that directly add pages to a bio > instead of using bio_iov_iter_get_pages(). > > Cc: Jens Axboe > Reviewed-by:

Re: [PATCH v13 02/42] iomap: support REQ_OP_ZONE_APPEND

2021-01-22 Thread Chaitanya Kulkarni
On 1/21/21 10:24 PM, Naohiro Aota wrote: > A ZONE_APPEND bio must follow hardware restrictions (e.g. not exceeding > max_zone_append_sectors) not to be split. bio_iov_iter_get_pages builds > such restricted bio using __bio_iov_append_get_pages if bio_op(bio) == > REQ_OP_ZONE_APPEND. > > To utilize

Re: [PATCH v13 01/42] block: add bio_add_zone_append_page

2021-01-22 Thread Chaitanya Kulkarni
On 1/21/21 10:24 PM, Naohiro Aota wrote: > From: Johannes Thumshirn > > Add bio_add_zone_append_page(), a wrapper around bio_add_hw_page() which > is intended to be used by file systems that directly add pages to a bio > instead of using bio_iov_iter_get_pages(). > > Cc: Jens Axboe > Reviewed-by:

Re: [RFC][PATCH V5] btrfs: preferred_metadata: preferred device for metadata

2021-01-22 Thread Zygo Blaxell
On Fri, Jan 22, 2021 at 07:31:18PM +0100, Goffredo Baroncelli wrote: > On 1/21/21 7:54 PM, Zygo Blaxell wrote: > > On Thu, Jan 21, 2021 at 07:16:05PM +0100, Goffredo Baroncelli wrote: > > > On 1/20/21 5:02 PM, Josef Bacik wrote: > > > > On 1/17/21 1:54 PM, Goffredo Baroncelli wrote: > > > > > > >

[PATCH man-pages v7] Document encoded I/O

2021-01-22 Thread Omar Sandoval
From: Omar Sandoval This adds a new page, encoded_io(7), providing an overview of encoded I/O and updates fcntl(2), open(2), and preadv2(2)/pwritev2(2) to reference it. Cc: Michael Kerrisk Cc: linux-man Signed-off-by: Omar Sandoval --- man2/fcntl.2 | 8 + man2/open.2 | 13 ++ m

[PATCH v7 01/10] iov_iter: add copy_struct_from_iter()

2021-01-22 Thread Omar Sandoval
From: Omar Sandoval This is essentially copy_struct_from_user() but for an iov_iter. Suggested-by: Aleksa Sarai Reviewed-by: Josef Bacik Signed-off-by: Omar Sandoval --- include/linux/uio.h | 2 ++ lib/iov_iter.c | 82 + 2 files changed, 84 i

[PATCH v7 02/10] fs: add O_ALLOW_ENCODED open flag

2021-01-22 Thread Omar Sandoval
From: Omar Sandoval The upcoming RWF_ENCODED operation introduces some security concerns: 1. Compressed writes will pass arbitrary data to decompression algorithms in the kernel. 2. Compressed reads can leak truncated/hole punched data. Therefore, we need to require privilege for RWF_ENCODED

[PATCH v7 05/10] btrfs: don't advance offset for compressed bios in btrfs_csum_one_bio()

2021-01-22 Thread Omar Sandoval
From: Omar Sandoval btrfs_csum_one_bio() loops over each filesystem block in the bio while keeping a cursor of its current logical position in the file in order to look up the ordered extent to add the checksums to. However, this doesn't make much sense for compressed extents, as a sector on disk

[PATCH v7 03/10] fs: add RWF_ENCODED for reading/writing compressed data

2021-01-22 Thread Omar Sandoval
From: Omar Sandoval Btrfs supports transparent compression: data written by the user can be compressed when written to disk and decompressed when read back. However, we'd like to add an interface to write pre-compressed data directly to the filesystem, and the matching interface to read compresse

[PATCH v7 07/10] btrfs: support different disk extent size for delalloc

2021-01-22 Thread Omar Sandoval
From: Omar Sandoval Currently, we always reserve the same extent size in the file and extent size on disk for delalloc because the former is the worst case for the latter. For RWF_ENCODED writes, we know the exact size of the extent on disk, which may be less than or greater than (for bookends) t

[PATCH v7 06/10] btrfs: add ram_bytes and offset to btrfs_ordered_extent

2021-01-22 Thread Omar Sandoval
From: Omar Sandoval Currently, we only create ordered extents when ram_bytes == num_bytes and offset == 0. However, RWF_ENCODED writes may create extents which only refer to a subset of the full unencoded extent, so we need to plumb these fields through the ordered extent infrastructure and pass

[PATCH v7 10/10] btrfs: implement RWF_ENCODED writes

2021-01-22 Thread Omar Sandoval
From: Omar Sandoval The implementation resembles direct I/O: we have to flush any ordered extents, invalidate the page cache, and do the io tree/delalloc/extent map/ordered extent dance. From there, we can reuse the compression code with a minor modification to distinguish the write from writebac

[PATCH v7 13/15] btrfs: send: allocate send buffer with alloc_page() and vmap() for v2

2021-01-22 Thread Omar Sandoval
From: Omar Sandoval For encoded writes, we need the raw pages for reading compressed data directly via a bio. So, replace kvmalloc() with vmap() so we have access to the raw pages. 144k is large enough that it usually gets allocated with vmalloc(), anyways. Signed-off-by: Omar Sandoval --- fs/

[PATCH v3 05/11] btrfs-progs: receive: add stub implementation for pwritev2

2021-01-22 Thread Omar Sandoval
From: Boris Burkov Encoded writes in receive will use pwritev2. It is possible that the system libc does not export this function, so we stub it out and detect whether to build the stub code with autoconf. This syscall has special semantics in x32 (no hi lo, just takes loff_t) so we have to dete

[PATCH v3 4/5] btrfs: send: send compressed extents with encoded writes

2021-01-22 Thread Omar Sandoval
From: Omar Sandoval Now that all of the pieces are in place, we can use the ENCODED_WRITE command to send compressed extents when appropriate. Signed-off-by: Omar Sandoval --- fs/btrfs/ctree.h | 4 + fs/btrfs/inode.c | 6 +- fs/btrfs/send.c | 230 ++

[PATCH v3 5/5] btrfs: send: enable support for stream v2 and compressed writes

2021-01-22 Thread Omar Sandoval
From: Omar Sandoval Now that the new support is implemented, allow the ioctl to accept the flags and update the version in sysfs. Signed-off-by: Omar Sandoval --- fs/btrfs/send.c| 10 +- fs/btrfs/send.h| 2 +- include/uapi/linux/btrfs.h | 4 +++- 3 files chang

[PATCH v3 06/11] btrfs-progs: receive: process encoded_write commands

2021-01-22 Thread Omar Sandoval
From: Boris Burkov Add a new btrfs_send_op and support for both dumping and proper receive processing which does actual encoded writes. Encoded writes are only allowed on a file descriptor opened with an extra flag that allows encoded writes, so we also add support for this flag when opening or

[PATCH v3 08/11] btrfs-progs: receive: process fallocate commands

2021-01-22 Thread Omar Sandoval
From: Boris Burkov Send stream v2 can emit fallocate commands, so receive must support them as well. The implementation simply passes along the arguments to the syscall. Note that mode is encoded as a u32 in send stream but fallocate takes an int, so there is a unsigned->signed conversion there.

[PATCH v3 07/11] btrfs-progs: receive: encoded_write fallback to explicit decode and write

2021-01-22 Thread Omar Sandoval
From: Boris Burkov An encoded_write can fail if the file system it is being applied to does not support encoded writes or if it can't find enough contiguous space to accommodate the encoded extent. In those cases, we can likely still process an encoded_write by explicitly decoding the data and do

[PATCH v3 09/11] btrfs-progs: receive: process setflags ioctl commands

2021-01-22 Thread Omar Sandoval
From: Boris Burkov In send stream v2, send can emit a command for setting inode flags via the setflags ioctl. Pass the flags attribute through to the ioctl call in receive. Signed-off-by: Boris Burkov --- cmds/receive-dump.c | 6 ++ cmds/receive.c | 24 com

[PATCH v3 10/11] btrfs-progs: send: stream v2 ioctl flags

2021-01-22 Thread Omar Sandoval
From: Boris Burkov To make the btrfs send ioctl use the stream v2 format requires passing BTRFS_SEND_FLAG_STREAM_V2 in flags. Further, to cause the ioctl to emit encoded_write commands for encoded extents, we must set that flag as well as BTRFS_SEND_FLAG_COMPRESSED. Finally, we bump up the versio

Re: [PATCH v7 11/15] btrfs: add send stream v2 definitions

2021-01-22 Thread Omar Sandoval
Please ignore 11-15. I fat fingered format-patch, these are part of the other series.

[PATCH v3 11/11] btrfs-progs: receive: add tests for basic encoded_write send/receive

2021-01-22 Thread Omar Sandoval
From: Boris Burkov Adapt the existing send/receive tests by passing '-o --force-compress' to the mount commands in a new test. After writing a few files in the various compression formats, send/receive them with and without --force-decompress to test both the encoded_write path and the fallback t

[PATCH v7 15/15] btrfs: send: enable support for stream v2 and compressed writes

2021-01-22 Thread Omar Sandoval
From: Omar Sandoval Now that the new support is implemented, allow the ioctl to accept the flags and update the version in sysfs. Signed-off-by: Omar Sandoval --- fs/btrfs/send.c| 10 +- fs/btrfs/send.h| 2 +- include/uapi/linux/btrfs.h | 4 +++- 3 files chang

[PATCH v3 3/5] btrfs: send: allocate send buffer with alloc_page() and vmap() for v2

2021-01-22 Thread Omar Sandoval
From: Omar Sandoval For encoded writes, we need the raw pages for reading compressed data directly via a bio. So, replace kvmalloc() with vmap() so we have access to the raw pages. 144k is large enough that it usually gets allocated with vmalloc(), anyways. Signed-off-by: Omar Sandoval --- fs/

[PATCH v3 04/11] btrfs-progs: receive: add send stream v2 cmds and attrs to send.h

2021-01-22 Thread Omar Sandoval
From: Boris Burkov Send stream v2 adds three commands and several attributes associated to those commands. Before we implement processing them, add all the commands and attributes. This avoids leaving the enums in an intermediate state that doesn't correspond to any version of send stream. Signe

[PATCH v3 03/11] btrfs-progs: receive: support v2 send stream DATA tlv format

2021-01-22 Thread Omar Sandoval
From: Boris Burkov The new format privileges the BTRFS_SEND_A_DATA attribute by guaranteeing it will always be the last attribute in any command that needs it, and by implicitly encoding the data length as the difference between the total command length in the command header and the sizes of the

[PATCH v3 2/5] btrfs: send: write larger chunks when using stream v2

2021-01-22 Thread Omar Sandoval
From: Omar Sandoval The length field of the send stream TLV header is 16 bits. This means that the maximum amount of data that can be sent for one write is 64k minus one. However, encoded writes must be able to send the maximum compressed extent (128k) in one command. To support this, send stream

[PATCH v3 02/11] btrfs-progs: receive: dynamically allocate sctx->read_buf

2021-01-22 Thread Omar Sandoval
From: Boris Burkov In send stream v2, write commands can now be an arbitrary size. For that reason, we can no longer allocate a fixed array in sctx for read_cmd. Instead, read_cmd dynamically allocates sctx->read_buf. To avoid needless reallocations, we reuse read_buf between read_cmd calls by al

[PATCH v7 14/15] btrfs: send: send compressed extents with encoded writes

2021-01-22 Thread Omar Sandoval
From: Omar Sandoval Now that all of the pieces are in place, we can use the ENCODED_WRITE command to send compressed extents when appropriate. Signed-off-by: Omar Sandoval --- fs/btrfs/ctree.h | 4 + fs/btrfs/inode.c | 6 +- fs/btrfs/send.c | 230 ++

[PATCH v3 01/11] btrfs-progs: receive: support v2 send stream larger tlv_len

2021-01-22 Thread Omar Sandoval
From: Boris Burkov An encoded extent can be up to 128K in length, which exceeds the largest value expressible by the current send stream format's 16 bit tlv_len field. Since encoded writes cannot be split into multiple writes by btrfs send, the send stream format must change to accommodate encode

[PATCH v3 0/5] btrfs: implement send/receive of compressed extents without decompressing

2021-01-22 Thread Omar Sandoval
From: Omar Sandoval This series uses the interface added in "fs: interface for directly reading/writing compressed data" to send and receive compressed data without wastefully decompressing and recompressing it. It does so by 1. Bumping the send stream protocol version to 2. 2. Adding a new comm

[PATCH v3 1/5] btrfs: add send stream v2 definitions

2021-01-22 Thread Omar Sandoval
From: Omar Sandoval This adds the definitions of the new commands for send stream version 2 and their respective attributes: fallocate, FS_IOC_SETFLAGS (a.k.a. chattr), and encoded writes. It also documents two changes to the send stream format in v2: the receiver shouldn't assume a maximum comma

[PATCH v7 11/15] btrfs: add send stream v2 definitions

2021-01-22 Thread Omar Sandoval
From: Omar Sandoval This adds the definitions of the new commands for send stream version 2 and their respective attributes: fallocate, FS_IOC_SETFLAGS (a.k.a. chattr), and encoded writes. It also documents two changes to the send stream format in v2: the receiver shouldn't assume a maximum comma

[PATCH v7 12/15] btrfs: send: write larger chunks when using stream v2

2021-01-22 Thread Omar Sandoval
From: Omar Sandoval The length field of the send stream TLV header is 16 bits. This means that the maximum amount of data that can be sent for one write is 64k minus one. However, encoded writes must be able to send the maximum compressed extent (128k) in one command. To support this, send stream

[PATCH v7 09/10] btrfs: implement RWF_ENCODED reads

2021-01-22 Thread Omar Sandoval
From: Omar Sandoval There are 4 main cases: 1. Inline extents: we copy the data straight out of the extent buffer. 2. Hole/preallocated extents: we fill in zeroes. 3. Regular, uncompressed extents: we read the sectors we need directly from disk. 4. Regular, compressed extents: we read the ent

[PATCH v7 08/10] btrfs: optionally extend i_size in cow_file_range_inline()

2021-01-22 Thread Omar Sandoval
From: Omar Sandoval Currently, an inline extent is always created after i_size is extended from btrfs_dirty_pages(). However, for encoded writes, we only want to update i_size after we successfully created the inline extent. Add an update_i_size parameter to cow_file_range_inline() and insert_inl

[PATCH v7 04/10] btrfs: fix check_data_csum() error message for direct I/O

2021-01-22 Thread Omar Sandoval
From: Omar Sandoval Commit 1dae796aabf6 ("btrfs: inode: sink parameter start and len to check_data_csum()") replaced the start parameter to check_data_csum() with page_offset(), but page_offset() is not meaningful for direct I/O pages. Bring back the start parameter. Fixes: 265d4ac03fdf ("btrfs:

[PATCH v7 00/10] fs: interface for directly reading/writing compressed data

2021-01-22 Thread Omar Sandoval
From: Omar Sandoval This series adds an API for reading compressed data on a filesystem without decompressing it as well as support for writing compressed data directly to the filesystem. As with the previous submissions, I've included a man page patch describing the API. I have test cases (inclu

Re: [PATCH] btrfs: fix log replay failure due to race with space cache rebuild

2021-01-22 Thread Filipe Manana
On Fri, Jan 22, 2021 at 6:39 PM Josef Bacik wrote: > > On 1/22/21 12:56 PM, fdman...@kernel.org wrote: > > From: Filipe Manana > > > > After a sudden power failure we may end up with a space cache on disk that > > is not valid and needs to be rebuilt from scratch. > > > > If that happens, during

Re: [PATCH v3] btrfs: fix log replay failure due to race with space cache rebuild

2021-01-22 Thread Josef Bacik
On 1/22/21 2:07 PM, fdman...@kernel.org wrote: From: Filipe Manana After a sudden power failure we may end up with a space cache on disk that is not valid and needs to be rebuilt from scratch. If that happens, during log replay when we attempt to pin an extent buffer from a log tree, at btrfs_

Re: [PATCH] btrfs: fix log replay failure due to race with space cache rebuild

2021-01-22 Thread Josef Bacik
On 1/22/21 12:56 PM, fdman...@kernel.org wrote: From: Filipe Manana After a sudden power failure we may end up with a space cache on disk that is not valid and needs to be rebuilt from scratch. If that happens, during log replay when we attempt to pin an extent buffer from a log tree, at btrfs

[PATCH v3] btrfs: fix log replay failure due to race with space cache rebuild

2021-01-22 Thread fdmanana
From: Filipe Manana After a sudden power failure we may end up with a space cache on disk that is not valid and needs to be rebuilt from scratch. If that happens, during log replay when we attempt to pin an extent buffer from a log tree, at btrfs_pin_extent_for_log_replay(), we do not wait for t

Re: [RFC][PATCH V5] btrfs: preferred_metadata: preferred device for metadata

2021-01-22 Thread Goffredo Baroncelli
On 1/21/21 7:54 PM, Zygo Blaxell wrote: On Thu, Jan 21, 2021 at 07:16:05PM +0100, Goffredo Baroncelli wrote: On 1/20/21 5:02 PM, Josef Bacik wrote: On 1/17/21 1:54 PM, Goffredo Baroncelli wrote: Hi all, This is an RFC; I wrote this patch because I find the idea interesting even though it add

[PATCH] btrfs: fix log replay failure due to race with space cache rebuild

2021-01-22 Thread fdmanana
From: Filipe Manana After a sudden power failure we may end up with a space cache on disk that is not valid and needs to be rebuilt from scratch. If that happens, during log replay when we attempt to pin an extent buffer from a log tree, at btrfs_pin_extent_for_log_replay(), we do not wait for t

Re: [PATCH 1/2] btrfs: fix log replay failure due to race with space cache rebuild

2021-01-22 Thread Filipe Manana
On Fri, Jan 22, 2021 at 4:43 PM Josef Bacik wrote: > > On 1/22/21 10:28 AM, fdman...@kernel.org wrote: > > From: Filipe Manana > > > > After a sudden power failure we may end up with a space cache on disk that > > is not valid and needs to be rebuilt from scratch. > > > > If that happens, during

Re: [PATCH 2/2] btrfs: fix log replay failure when space cache needs to be rebuilt

2021-01-22 Thread Josef Bacik
On 1/22/21 10:28 AM, fdman...@kernel.org wrote: From: Filipe Manana During log replay we first start by walking the log trees and pin the ranges for their extent buffers, through calls to the function btrfs_pin_extent_for_log_replay(). However if the space cache for a block group is invalid an

Re: [PATCH 1/2] btrfs: fix log replay failure due to race with space cache rebuild

2021-01-22 Thread Josef Bacik
On 1/22/21 10:28 AM, fdman...@kernel.org wrote: From: Filipe Manana After a sudden power failure we may end up with a space cache on disk that is not valid and needs to be rebuilt from scratch. If that happens, during log replay when we attempt to pin an extent buffer from a log tree, at btrfs

Re: [PATCH v3 14/14] btrfs: Enable W=1 checks for btrfs

2021-01-22 Thread David Sterba
On Fri, Jan 22, 2021 at 11:58:05AM +0200, Nikolay Borisov wrote: > Now that the btrfs' codebase is clean of W=1 warning let's enable those > checks unconditionally for the entire fs/btrfs/ and its subdirectories. > > Signed-off-by: Nikolay Borisov > --- > fs/btrfs/Makefile | 17 +

Re: [PATCH v13 41/42] btrfs: serialize log transaction on ZONED mode

2021-01-22 Thread Josef Bacik
On 1/22/21 1:21 AM, Naohiro Aota wrote: This is the 2/3 patch to enable tree-log on ZONED mode. Since we can start more than one log transactions per subvolume simultaneously, nodes from multiple transactions can be allocated interleaved. Such mixed allocation results in non-sequential writes at

[PATCH 1/2] btrfs: fix log replay failure due to race with space cache rebuild

2021-01-22 Thread fdmanana
From: Filipe Manana After a sudden power failure we may end up with a space cache on disk that is not valid and needs to be rebuilt from scratch. If that happens, during log replay when we attempt to pin an extent buffer from a log tree, at btrfs_pin_extent_for_log_replay(), we do not wait for t

Re: [PATCH v13 27/42] btrfs: use ZONE_APPEND write for ZONED btrfs

2021-01-22 Thread Josef Bacik
On 1/22/21 1:21 AM, Naohiro Aota wrote: This commit enables zone append writing for zoned btrfs. When using zone append, a bio is issued to the start of a target zone and the device decides to place it inside the zone. Upon completion the device reports the actual written position back to the hos

[PATCH 2/2] btrfs: fix log replay failure when space cache needs to be rebuilt

2021-01-22 Thread fdmanana
From: Filipe Manana During log replay we first start by walking the log trees and pin the ranges for their extent buffers, through calls to the function btrfs_pin_extent_for_log_replay(). However if the space cache for a block group is invalid and needs to be rebuilt, we can fail the log replay

[PATCH 0/2] btrfs: a couple bug fixes for failures of log replay and mount

2021-01-22 Thread fdmanana
From: Filipe Manana This small patchset fixes two bugs that lead to an -EINVAL failure during log replay, causing the filesystem mount to fail. They are relatively new regressions, one caused by the recent change to make space cache loading asynchronous and the other caused by the refactoring tha

Re: [PATCH v13 05/42] btrfs: release path before calling into btrfs_load_block_group_zone_info

2021-01-22 Thread Josef Bacik
On 1/22/21 1:21 AM, Naohiro Aota wrote: From: Johannes Thumshirn Since we have no write pointer in conventional zones, we cannot determine the allocation offset from it. Instead, we set the allocation offset after the highest addressed extent. This is done by reading the extent tree in btrfs_lo

Re: [PATCH v13 26/42] btrfs: save irq flags when looking up an ordered extent

2021-01-22 Thread Josef Bacik
On 1/22/21 1:21 AM, Naohiro Aota wrote: From: Johannes Thumshirn A following patch will add another caller of btrfs_lookup_ordered_extent() from a bio endio context. btrfs_lookup_ordered_extent() uses spin_lock_irq() which unconditionally disables interrupts. Change this to spin_lock_irqsave()

Re: [PATCH v13 22/42] btrfs: split ordered extent when bio is sent

2021-01-22 Thread Josef Bacik
On 1/22/21 1:21 AM, Naohiro Aota wrote: For a zone append write, the device decides the location the data is written to. Therefore we cannot ensure that two bios are written consecutively on the device. In order to ensure that a ordered extent maps to a contiguous region on disk, we need to maint

Re: [PATCH v13 23/42] btrfs: check if bio spans across an ordered extent

2021-01-22 Thread Josef Bacik
On 1/22/21 1:21 AM, Naohiro Aota wrote: From: Johannes Thumshirn To ensure that an ordered extent maps to a contiguous region on disk, we need to maintain a "one bio == one ordered extent" rule. This commit ensures that constructing bio does not span across an ordered extent. Signed-off-by: J

Re: [PATCH v13 13/42] btrfs: track unusable bytes for zones

2021-01-22 Thread Josef Bacik
On 1/22/21 1:21 AM, Naohiro Aota wrote: In zoned btrfs a region that was once written then freed is not usable until we reset the underlying zone. So we need to distinguish such unusable space from usable free space. Therefore we need to introduce the "zone_unusable" field to the block group st

Re: [PATCH v13 12/42] btrfs: calculate allocation offset for conventional zones

2021-01-22 Thread Josef Bacik
On 1/22/21 1:21 AM, Naohiro Aota wrote: Conventional zones do not have a write pointer, so we cannot use it to determine the allocation offset if a block group contains a conventional zone. But instead, we can consider the end of the last allocated extent in the block group as an allocation offs

Re: [PATCH v13 08/42] btrfs: allow zoned mode on non-zoned block devices

2021-01-22 Thread Josef Bacik
On 1/22/21 1:21 AM, Naohiro Aota wrote: From: Johannes Thumshirn Run zoned btrfs mode on non-zoned devices. This is done by "slicing up" the block-device into static sized chunks and fake a conventional zone on each of them. The emulated zone size is determined from the size of device extent.

Re: [PATCH v13 07/42] btrfs: disallow fitrim in ZONED mode

2021-01-22 Thread Josef Bacik
On 1/22/21 1:21 AM, Naohiro Aota wrote: The implementation of fitrim is depending on space cache, which is not used and disabled for zoned btrfs' extent allocator. So the current code does not work with zoned btrfs. In the future, we can implement fitrim for zoned btrfs by enabling space cache (b

Re: [PATCH v13 06/42] btrfs: do not load fs_info->zoned from incompat flag

2021-01-22 Thread Josef Bacik
On 1/22/21 1:21 AM, Naohiro Aota wrote: From: Johannes Thumshirn Don't set the zoned flag in fs_info when encountering the BTRFS_FEATURE_INCOMPAT_ZONED on mount. The zoned flag in fs_info is in a union together with the zone_size, so setting it too early will result in setting an incorrect zone

Re: [PATCH v3 04/14] btrfs: Fix parameter description in delayed-ref.c functions

2021-01-22 Thread Johannes Thumshirn
Looks good, Reviewed-by: Johannes Thumshirn

Re: [PATCH v3 03/14] btrfs: Fix function description format

2021-01-22 Thread Johannes Thumshirn
On 22/01/2021 11:17, Nikolay Borisov wrote: > This fixes following W=1 warnings: > > fs/btrfs/file-item.c:27: warning: Cannot understand * @inode: the inode we > want to update the disk_i_size for > on line 27 - I thought it was a doc line > fs/btrfs/file-item.c:65: warning: Cannot understand

Re: [PATCH v3 02/14] btrfs: Fix parameter description of btrfs_add_extent_mapping

2021-01-22 Thread Johannes Thumshirn
Looks good, Reviewed-by: Johannes Thumshirn

Re: [PATCH v3 01/14] btrfs: Document modified parameter of add_extent_mapping

2021-01-22 Thread Johannes Thumshirn
On 22/01/2021 11:17, Nikolay Borisov wrote: > Fixes fs/btrfs/extent_map.c:399: warning: Function parameter or member > 'modified' not described in 'add_extent_mapping' > > Signed-off-by: Nikolay Borisov > --- > fs/btrfs/extent_map.c | 5 - > 1 file changed, 4 insertions(+), 1 deletion(-) >

Re: [PATCH v3 13/14] lib/zstd: Convert constants to defines

2021-01-22 Thread Nikolay Borisov
Nick, Can we get your ACK for the below changes. THey don't seem to be used outside of core zstd code, yet they result in warnings in code which includes zstd.h. By switching them to defines we don't lose anything. On 22.01.21 г. 11:58 ч., Nikolay Borisov wrote: > Those constants are really used

[PATCH v3 00/14] Make btrfs W=1 clean

2021-01-22 Thread Nikolay Borisov
Reworked patches to make touched kdocs adhere to the style described at https://btrfs.wiki.kernel.org/index.php/Development_notes#Coding_style_preferences Namely: * Removed function names from first line and left only short description * Aligned all arguments Nikolay Borisov (14): btrfs: Do

[PATCH v3 04/14] btrfs: Fix parameter description in delayed-ref.c functions

2021-01-22 Thread Nikolay Borisov
This fixes the following warnings: fs/btrfs/delayed-ref.c:80: warning: Function parameter or member 'fs_info' not described in 'btrfs_delayed_refs_rsv_release' fs/btrfs/delayed-ref.c:80: warning: Function parameter or member 'nr' not described in 'btrfs_delayed_refs_rsv_release' fs/btrfs/delayed

[PATCH v3 03/14] btrfs: Fix function description format

2021-01-22 Thread Nikolay Borisov
This fixes following W=1 warnings: fs/btrfs/file-item.c:27: warning: Cannot understand * @inode: the inode we want to update the disk_i_size for on line 27 - I thought it was a doc line fs/btrfs/file-item.c:65: warning: Cannot understand * @inode - the inode we're modifying on line 65 - I t

[PATCH v3 01/14] btrfs: Document modified parameter of add_extent_mapping

2021-01-22 Thread Nikolay Borisov
Fixes fs/btrfs/extent_map.c:399: warning: Function parameter or member 'modified' not described in 'add_extent_mapping' Signed-off-by: Nikolay Borisov --- fs/btrfs/extent_map.c | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/extent_map.c b/fs/btrfs/extent_map.c i

[PATCH v3 02/14] btrfs: Fix parameter description of btrfs_add_extent_mapping

2021-01-22 Thread Nikolay Borisov
This fixes the following compiler warnings: fs/btrfs/extent_map.c:601: warning: Function parameter or member 'fs_info' not described in 'btrfs_add_extent_mapping' fs/btrfs/extent_map.c:601: warning: Function parameter or member 'em_tree' not described in 'btrfs_add_extent_mapping' fs/btrfs/exten

[PATCH v3 13/14] lib/zstd: Convert constants to defines

2021-01-22 Thread Nikolay Borisov
Those constants are really used internally by zstd and including linux/zstd.h into users results in the following warnings: In file included from fs/btrfs/zstd.c:19: ./include/linux/zstd.h:798:21: warning: ‘ZSTD_skippableHeaderSize’ defined but not used [-Wunused-const-variable=] 798 | static c

[PATCH v3 14/14] btrfs: Enable W=1 checks for btrfs

2021-01-22 Thread Nikolay Borisov
Now that the btrfs' codebase is clean of W=1 warning let's enable those checks unconditionally for the entire fs/btrfs/ and its subdirectories. Signed-off-by: Nikolay Borisov --- fs/btrfs/Makefile | 17 + 1 file changed, 17 insertions(+) diff --git a/fs/btrfs/Makefile b/fs/btrfs

[PATCH v3 09/14] btrfs: Document btrfs_check_shared parameters

2021-01-22 Thread Nikolay Borisov
Signed-off-by: Nikolay Borisov --- fs/btrfs/backref.c | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/backref.c b/fs/btrfs/backref.c index ef71aba5bc15..bbe50affb554 100644 --- a/fs/btrfs/backref.c +++ b/fs/btrfs/backref.c @@ -1501,7 +1501,13 @@ int btrfs_find_

[PATCH v3 07/14] btrfs: Document fs_info in btrfs_rmap_block

2021-01-22 Thread Nikolay Borisov
Fixes fs/btrfs/block-group.c:1570: warning: Function parameter or member 'fs_info' not described in 'btrfs_rmap_block' Signed-off-by: Nikolay Borisov --- fs/btrfs/block-group.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c

[PATCH v3 06/14] btrfs: Document now parameter of peek_discard_list

2021-01-22 Thread Nikolay Borisov
Fixes fs/btrfs/discard.c:203: warning: Function parameter or member 'now' not described in 'peek_discard_list' Signed-off-by: Nikolay Borisov --- fs/btrfs/discard.c | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/discard.c b/fs/btrfs/discard.c index 2b8383d411

[PATCH v3 08/14] btrfs: Fix description format of fs_info parameter of btrfs_wait_on_delayed_iputs

2021-01-22 Thread Nikolay Borisov
Fixes fs/btrfs/inode.c:3101: warning: Function parameter or member 'fs_info' not described in 'btrfs_wait_on_delayed_iputs' Signed-off-by: Nikolay Borisov --- fs/btrfs/inode.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 5906

[PATCH v3 05/14] btrfs: Improve parameter description for __btrfs_write_out_cache

2021-01-22 Thread Nikolay Borisov
Fixes following W=1 warnings: fs/btrfs/free-space-cache.c:1317: warning: Function parameter or member 'root' not described in '__btrfs_write_out_cache' fs/btrfs/free-space-cache.c:1317: warning: Function parameter or member 'inode' not described in '__btrfs_write_out_cache' fs/btrfs/free-space-ca

[PATCH v3 11/14] btrfs: Fix parameter description in space-info.c

2021-01-22 Thread Nikolay Borisov
With these fixes space-info.c is clearn for W=1 warnings, namely the following ones are fixed: fs/btrfs/space-info.c:575: warning: Function parameter or member 'fs_info' not described in 'may_commit_transaction' fs/btrfs/space-info.c:575: warning: Function parameter or member 'space_info' not de

[PATCH v3 10/14] btrfs: Fix parameter description of btrfs_inode_rsv_release/btrfs_delalloc_release_space

2021-01-22 Thread Nikolay Borisov
Fixes following warnings: fs/btrfs/delalloc-space.c:205: warning: Function parameter or member 'inode' not described in 'btrfs_inode_rsv_release' fs/btrfs/delalloc-space.c:205: warning: Function parameter or member 'qgroup_free' not described in 'btrfs_inode_rsv_release' fs/btrfs/delalloc-space.

[PATCH v3 12/14] btrfs: Fix parameter description for functions in extent_io.c

2021-01-22 Thread Nikolay Borisov
This makes the file W=1 clean and fixes the following warnings: fs/btrfs/extent_io.c:414: warning: Function parameter or member 'tree' not described in '__etree_search' fs/btrfs/extent_io.c:414: warning: Function parameter or member 'offset' not described in '__etree_search' fs/btrfs/extent_io.c

Re: [PATCH] btrfs: make Private2 lifespan more consistent

2021-01-22 Thread Qu Wenruo
On 2021/1/22 下午4:24, Qu Wenruo wrote: On 2021/1/22 下午4:04, Nikolay Borisov wrote: On 22.01.21 г. 8:00 ч., Qu Wenruo wrote: Currently btrfs uses page Private2 bit to incidate if we have ordered extent for the page range. But the lifespan of it is not consistent, during regular writeback

Re: [PATCH] btrfs: make Private2 lifespan more consistent

2021-01-22 Thread Qu Wenruo
On 2021/1/22 下午4:04, Nikolay Borisov wrote: On 22.01.21 г. 8:00 ч., Qu Wenruo wrote: Currently btrfs uses page Private2 bit to incidate if we have ordered extent for the page range. But the lifespan of it is not consistent, during regular writeback path, there are two locations to clear th

Re: [PATCH v4 1/3] btrfs: add read_policy latency

2021-01-22 Thread Anand Jain
On 22/1/21 1:52 am, David Sterba wrote: On Thu, Jan 21, 2021 at 06:10:36PM +0800, Anand Jain wrote: On 20/1/21 8:14 pm, David Sterba wrote: On Tue, Jan 19, 2021 at 11:52:05PM -0800, Anand Jain wrote: The read policy type latency routes the read IO based on the historical average wait-time

Re: [PATCH] btrfs: make Private2 lifespan more consistent

2021-01-22 Thread Nikolay Borisov
On 22.01.21 г. 8:00 ч., Qu Wenruo wrote: > Currently btrfs uses page Private2 bit to incidate if we have ordered > extent for the page range. > > But the lifespan of it is not consistent, during regular writeback path, > there are two locations to clear the same PagePrivate2: > > T - P