[PATCH] btrfs: Simplify the calculation of variables

2021-01-27 Thread Abaci Team
Fix the following coccicheck warnings: ./fs/btrfs/delayed-inode.c:1157:39-41: WARNING !A || A && B is equivalent to !A || B. Reported-by: Abaci Robot Suggested-by: Jiapeng Zhong Signed-off-by: Abaci Team --- fs/btrfs/delayed-inode.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff

Re: btrfs becomes read-only

2021-01-27 Thread Alexey Isaev
kernel version: aleksey@host:~$ sudo uname --all Linux host 4.15.0-132-generic #136~16.04.1-Ubuntu SMP Tue Jan 12 18:22:20 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux drive make/model: Drive is external 5 bay HDD enclosure with raid-5 connected via usb-3 (made by Orico https://www.orico.cc/us/pr

[PATCH] fs: btrfs: Select SHA256 in Kconfig

2021-01-27 Thread matthias . bgg
From: Matthias Brugger Since commit 565a4147d17a ("fs: btrfs: Add more checksum algorithms") btrfs uses the sha256 checksum algorithm. But Kconfig lacks to select it. This leads to compilation errors: fs/built-in.o: In function `hash_sha256': fs/btrfs/crypto/hash.c:25: undefined reference to `sha

Re: [PATCH] fs: btrfs: Select SHA256 in Kconfig

2021-01-27 Thread Qu Wenruo
On 2021/1/27 下午5:42, matthias@kernel.org wrote: From: Matthias Brugger Since commit 565a4147d17a ("fs: btrfs: Add more checksum algorithms") btrfs uses the sha256 checksum algorithm. But Kconfig lacks to select it. This leads to compilation errors: fs/built-in.o: In function `hash_sha256

[PATCH] btrfs: Avoid calling btrfs_get_chunk_map() twice

2021-01-27 Thread Michal Rostecki
From: Michal Rostecki Before this change, the btrfs_get_io_geometry() function was calling btrfs_get_chunk_map() to get the extent mapping, necessary for calculating the I/O geometry. It was using that extent mapping only internally and freeing the pointer after its execution. That resulted in c

Re: [PATCH 03/17] blk-crypto: use bio_kmalloc in blk_crypto_clone_bio

2021-01-27 Thread Eric Biggers
On Tue, Jan 26, 2021 at 03:52:33PM +0100, Christoph Hellwig wrote: > Use bio_kmalloc instead of open coding it. > > Signed-off-by: Christoph Hellwig > --- > block/blk-crypto-fallback.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/block/blk-crypto-fallback.c b/block/b

[PATCH v4 11/12] btrfs: adjust the flush trace point to include the source

2021-01-27 Thread Josef Bacik
Since we have normal ticketed flushing and preemptive flushing, adjust the tracepoint so that we know the source of the flushing action to make it easier to debug problems. Reviewed-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c| 20 includ

[PATCH v4 08/12] btrfs: rework btrfs_calc_reclaim_metadata_size

2021-01-27 Thread Josef Bacik
Currently btrfs_calc_reclaim_metadata_size does two things, it returns the space currently required for flushing by the tickets, and if there are no tickets it calculates a value for the preemptive flushing. However for the normal ticketed flushing we really only care about the space required for

[PATCH v4 09/12] btrfs: simplify the logic in need_preemptive_flushing

2021-01-27 Thread Josef Bacik
A lot of this was added all in one go with no explanation, and is a bit unwieldy and confusing. Simplify the logic to start preemptive flushing if we've reserved more than half of our available free space. Reviewed-by: Nikolay Borisov Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 73 +

[PATCH v4 02/12] btrfs: add a trace point for reserve tickets

2021-01-27 Thread Josef Bacik
While debugging a ENOSPC related performance problem I needed to see the time difference between start and end of a reserve ticket, so add a trace point to report when we handle a reserve ticket. I opted to spit out start_ns itself without calculating the difference because there could be a gap be

[PATCH v4 04/12] btrfs: introduce a FORCE_COMMIT_TRANS flush operation

2021-01-27 Thread Josef Bacik
Sole-y for preemptive flushing, we want to be able to force the transaction commit without any of the ambiguity of may_commit_transaction(). This is because may_commit_transaction() checks tickets and such, and in preemptive flushing we already know it'll be helpful, so use this to keep the code n

Re: [PATCH v3 01/12] btrfs: make flush_space take a enum btrfs_flush_state instead of int

2021-01-27 Thread Josef Bacik
On 1/26/21 1:36 PM, David Sterba wrote: On Fri, Oct 09, 2020 at 09:28:18AM -0400, Josef Bacik wrote: I got a automated message from somebody who runs clang against our kernels and it's because I used the wrong enum type for what I passed into flush_space. Change the argument to be explicitly th

[PATCH v4 00/12] Improve preemptive ENOSPC flushing

2021-01-27 Thread Josef Bacik
v3->v4: - Added some comments. - Updated the tracepoint for the preemptive flushing to take a bool instead of an int. - Removed 'inline' from need_preemptive_flushing. v2->v3: - Added a cleanup to make sure we pass the right enum around for flush_space(). - Fixed a problem reported by a clang bu

[PATCH v4 01/12] btrfs: make flush_space take a enum btrfs_flush_state instead of int

2021-01-27 Thread Josef Bacik
I got a automated message from somebody who runs clang against our kernels and it's because I used the wrong enum type for what I passed into flush_space. Change the argument to be explicitly the enum we're expecting to make everything consistent. Maybe eventually gcc will catch errors like this.

Re: [PATCH] btrfs: send: use struct send_ctx *sctx for btrfs_compare_trees and changed_cb

2021-01-27 Thread David Sterba
On Mon, Jan 25, 2021 at 08:43:25PM +0100, Roman Anasal wrote: > btrfs_compare_trees and changed_cb use a void *ctx parameter instead of > struct send_ctx *sctx but when used in changed_cb it is immediately > casted to `struct send_ctx *sctx = ctx;`. > > changed_cb is only ever called from btrfs_co

Re: [PATCH v3 06/12] btrfs: rename need_do_async_reclaim

2021-01-27 Thread David Sterba
On Fri, Oct 09, 2020 at 09:28:23AM -0400, Josef Bacik wrote: > All of our normal flushing is asynchronous reclaim, so this helper is > poorly named. This is more checking if we need to preemptively flush > space, so rename it to need_preemptive_reclaim. > > Reviewed-by: Nikolay Borisov > Signed-

Re: [PATCH 02/17] btrfs: use bio_kmalloc in __alloc_device

2021-01-27 Thread Chaitanya Kulkarni
On 1/26/21 7:04 AM, Christoph Hellwig wrote: > Use bio_kmalloc instead of open coding it. > > Signed-off-by: Christoph Hellwig Looks good. Reviewed-by: Chaitanya Kulkarni

Re: [PATCH 01/17] zonefs: use bio_alloc in zonefs_file_dio_append

2021-01-27 Thread Chaitanya Kulkarni
On 1/26/21 7:01 AM, Christoph Hellwig wrote: > Use bio_alloc instead of open coding it. > > Signed-off-by: Christoph Hellwig Ha Ha I was going send out a patch for this :P. Reviewed-by: Chaitanya Kulkarni

Re: [PATCH v3 11/12] btrfs: adjust the flush trace point to include the source

2021-01-27 Thread David Sterba
On Fri, Oct 09, 2020 at 09:28:28AM -0400, Josef Bacik wrote: > Since we have normal ticketed flushing and preemptive flushing, adjust > the tracepoint so that we know the source of the flushing action to make > it easier to debug problems. > > Reviewed-by: Nikolay Borisov > Signed-off-by: Josef B

Re: [PATCH 05/17] block: use an on-stack bio in blkdev_issue_flush

2021-01-27 Thread Chaitanya Kulkarni
On 1/26/21 7:07 AM, Christoph Hellwig wrote: > There is no point in allocating memory for a synchronous flush. > > Signed-off-by: Christoph Hellwig True, looks good. Reviewed-by: Chaitanya Kulkarni

Re: [PATCH 16/17] nilfs2: remove cruft in nilfs_alloc_seg_bio

2021-01-27 Thread Chaitanya Kulkarni
On 1/26/21 7:27 AM, Christoph Hellwig wrote: > bio_alloc never returns NULL when it can sleep. > > Signed-off-by: Christoph Hellwig > --- Looks good. Reviewed-by: Chaitanya Kulkarni

Re: [PATCH v2 3/4] btrfs: send: fix invalid commands for inodes with changed rdev but same gen

2021-01-27 Thread Roman Anasal | BDSU
Am Montag, den 25.01.2021, 20:51 + schrieb Filipe Manana: > On Mon, Jan 25, 2021 at 7:51 PM Roman Anasal > wrote: > > This is analogous to the preceding patch ("btrfs: send: fix invalid > > commands for inodes with changed type but same gen") but for > > changed > > rdev: > > > > When doing a

Re: [PATCH v3 02/12] btrfs: add a trace point for reserve tickets

2021-01-27 Thread David Sterba
On Fri, Oct 09, 2020 at 09:28:19AM -0400, Josef Bacik wrote: > While debugging a ENOSPC related performance problem I needed to see the > time difference between start and end of a reserve ticket, so add a > trace point to report when we handle a reserve ticket. > > I opted to spit out start_ns it

Re: [PATCH v5 0/8] A variety of lock contention fixes

2021-01-27 Thread David Sterba
On Fri, Dec 18, 2020 at 02:24:18PM -0500, Josef Bacik wrote: > v4->v5: > - Added "btrfs: remove bogus BUG_ON in alloc_reserved_tree_block", as Nikolay > pointed out I needed to explain why we no longer needed one of the delayed > ref > flushes, which led me down the rabbit hole of trying to un

Re: [PATCH v5 2/8] btrfs: only let one thread pre-flush delayed refs in commit

2021-01-27 Thread David Sterba
On Tue, Jan 12, 2021 at 11:17:45AM +0200, Nikolay Borisov wrote: > > > On 11.01.21 г. 23:50 ч., David Sterba wrote: > > On Mon, Jan 11, 2021 at 10:33:42AM +0200, Nikolay Borisov wrote: > >> On 8.01.21 г. 18:01 ч., David Sterba wrote: > >>> On Fri, Dec 18, 2020 at 02:24:20PM -0500, Josef Bacik wro

[PATCH 3/7] btrfs: avoid logging new ancestor inodes when logging new inode

2021-01-27 Thread fdmanana
From: Filipe Manana When we fsync a new file, created in the current transaction, we check all its ancestor inodes and always log them if they were created in the current transaction - even if we have already logged them before, which is a waste of time. So avoid logging new ancestor inodes if t

[PATCH 4/7] btrfs: skip logging directories already logged when logging all parents

2021-01-27 Thread fdmanana
From: Filipe Manana Some times when we fsync an inode we need to do a full log of all its ancestors (due to unlink, link or rename operations), which can be an expensive operation, specially if the directories are large. However if we find an ancestor directory inode that is already logged in th

[PATCH 0/7] btrfs: more performance improvements for dbench workloads

2021-01-27 Thread fdmanana
From: Filipe Manana The following patchset brings one more batch of performance improvements with dbench workloads, or anything that mixes file creation, file writes, renames, unlinks, etc with fsync like dbench does. This patchset is mostly based on avoiding logging directory inodes multiple tim

[PATCH 7/7] btrfs: make concurrent fsyncs wait less when waiting for a transaction commit

2021-01-27 Thread fdmanana
From: Filipe Manana Often an fsync needs to fallback to a transaction commit for several reasons (to ensure consistency after a power failure, a new block group was allocated or a temporary error such as ENOMEM or ENOSPC happened). In that case the log is marked as needing a full commit and any

[PATCH 1/7] btrfs: remove unnecessary directory inode item update when deleting dir entry

2021-01-27 Thread fdmanana
From: Filipe Manana When we remove a directory entry, as part of an unlink operation, if the directory was logged before we must remove the directory index items from the log. We are also updating the inode item of the directory to update its i_size, but that is not necessary because during log r

[PATCH 5/7] btrfs: skip logging inodes already logged when logging new entries

2021-01-27 Thread fdmanana
From: Filipe Manana When logging new directory entries of a directory, we log the inodes of new dentries and the inodes of dentries pointing to directories that may have been created in past transactions. For the case of directories we log in full mode, which can be particularly expensive for lar

[PATCH 2/7] btrfs: stop setting nbytes when filling inode item for logging

2021-01-27 Thread fdmanana
From: Filipe Manana When we fill an inode item for logging we are setting its nbytes field with the value returned by inode_get_bytes() (a VFS API), however we do not need it because it is not used during log replay. In fact, for fast fsyncs, when we call inode_get_bytes() we may even get an outd

[PATCH 6/7] btrfs: remove unnecessary check_parent_dirs_for_sync()

2021-01-27 Thread fdmanana
From: Filipe Manana Whenever we fsync an inode, if it is a directory, a regular file that was created in the current transaction or has last_unlink_trans set to the generation of the current transaction, we check if any of its ancestor inodes (and the inode itself if it is a directory) can not be

Re: [PATCH] btrfs: fix a bug that btrfs_invalidapge() can double account ordered extent for subpage

2021-01-27 Thread Filipe Manana
On Wed, Jan 27, 2021 at 8:29 AM Qu Wenruo wrote: > > Commit dbfdb6d1b369 ("Btrfs: Search for all ordered extents that could > span across a page") make btrfs_invalidapage() to search all ordered > extents. > > The offending code looks like this: > > again: > start = page_start; > o

Re: [PATCH] btrfs: fix a bug that btrfs_invalidapge() can double account ordered extent for subpage

2021-01-27 Thread Qu Wenruo
On 2021/1/27 下午6:44, Filipe Manana wrote: On Wed, Jan 27, 2021 at 8:29 AM Qu Wenruo wrote: Commit dbfdb6d1b369 ("Btrfs: Search for all ordered extents that could span across a page") make btrfs_invalidapage() to search all ordered extents. The offending code looks like this: again:

Re: [PATCH v2 3/4] btrfs: send: fix invalid commands for inodes with changed rdev but same gen

2021-01-27 Thread Filipe Manana
On Tue, Jan 26, 2021 at 7:19 PM Roman Anasal | BDSU wrote: > > Am Montag, den 25.01.2021, 20:51 + schrieb Filipe Manana: > > On Mon, Jan 25, 2021 at 7:51 PM Roman Anasal > > wrote: > > > This is analogous to the preceding patch ("btrfs: send: fix invalid > > > commands for inodes with changed

Re: [PATCH] btrfs: Avoid calling btrfs_get_chunk_map() twice

2021-01-27 Thread Filipe Manana
On Wed, Jan 27, 2021 at 9:59 AM Michal Rostecki wrote: > > From: Michal Rostecki > > Before this change, the btrfs_get_io_geometry() function was calling > btrfs_get_chunk_map() to get the extent mapping, necessary for > calculating the I/O geometry. It was using that extent mapping only > intern

Re: [PATCH] fs: btrfs: Select SHA256 in Kconfig

2021-01-27 Thread David Sterba
On Wed, Jan 27, 2021 at 10:42:30AM +0100, matthias@kernel.org wrote: > From: Matthias Brugger > > Since commit 565a4147d17a ("fs: btrfs: Add more checksum algorithms") > btrfs uses the sha256 checksum algorithm. But Kconfig lacks to select > it. This leads to compilation errors: > fs/built-in

Re: [PATCH] btrfs: Avoid calling btrfs_get_chunk_map() twice

2021-01-27 Thread Michal Rostecki
On Wed, Jan 27, 2021 at 11:20:55AM +, Filipe Manana wrote: > On Wed, Jan 27, 2021 at 9:59 AM Michal Rostecki wrote: > > > > From: Michal Rostecki > > > > Before this change, the btrfs_get_io_geometry() function was calling > > btrfs_get_chunk_map() to get the extent mapping, necessary for > >

Re: [PATCH] fs: btrfs: Select SHA256 in Kconfig

2021-01-27 Thread Qu Wenruo
On 2021/1/27 下午8:01, David Sterba wrote: On Wed, Jan 27, 2021 at 10:42:30AM +0100, matthias@kernel.org wrote: From: Matthias Brugger Since commit 565a4147d17a ("fs: btrfs: Add more checksum algorithms") btrfs uses the sha256 checksum algorithm. But Kconfig lacks to select it. This leads

Re: Only one subvolume can be mounted after replace/balance

2021-01-27 Thread Jakob Schöttl
Thank you Chris, it's resolved now, see below. Am 25.01.21 um 23:47 schrieb Chris Murphy: On Sat, Jan 23, 2021 at 7:50 AM Jakob Schöttl wrote: Hi, In short: When mounting a second subvolume from a pool, I get this error: "mount: /mnt: wrong fs type, bad option, bad superblock on /dev/sda, mis

Re: btrfs becomes read-only

2021-01-27 Thread Alexey Isaev
I managed to run btrs check, but it didn't solve the problem: aleksey@host:~$ sudo btrfs check --repair /dev/sdg [sudo] password for aleksey: enabling repair mode Checking filesystem on /dev/sdg UUID: 070ce9af-6511-4b89-a501-0823514320c1 checking extents parent transid verify failed on 5218004833

Re: [PATCH] fs: btrfs: Select SHA256 in Kconfig

2021-01-27 Thread David Sterba
On Wed, Jan 27, 2021 at 08:14:31PM +0800, Qu Wenruo wrote: > > > On 2021/1/27 下午8:01, David Sterba wrote: > > On Wed, Jan 27, 2021 at 10:42:30AM +0100, matthias@kernel.org wrote: > >> From: Matthias Brugger > >> > >> Since commit 565a4147d17a ("fs: btrfs: Add more checksum algorithms") > >>

Re: [PATCH] fs: btrfs: Select SHA256 in Kconfig

2021-01-27 Thread Marek Behun
On Wed, 27 Jan 2021 14:47:58 +0100 David Sterba wrote: > If it's a series then please mention u-boot in the cover letter, no need > to change the patches, I'll go check CC if I'm too confused about the > patch. I would suggest using subject prefix [PATCH u-boot]

[PATCH v2] btrfs: Avoid calling btrfs_get_chunk_map() twice

2021-01-27 Thread Michal Rostecki
From: Michal Rostecki Before this change, the btrfs_get_io_geometry() function was calling btrfs_get_chunk_map() to get the extent mapping, necessary for calculating the I/O geometry. It was using that extent mapping only internally and freeing the pointer after its execution. That resulted in c

Re: [PATCH v2] btrfs: Avoid calling btrfs_get_chunk_map() twice

2021-01-27 Thread Nikolay Borisov
On 27.01.21 г. 15:57 ч., Michal Rostecki wrote: > From: Michal Rostecki > > Before this change, the btrfs_get_io_geometry() function was calling > btrfs_get_chunk_map() to get the extent mapping, necessary for > calculating the I/O geometry. It was using that extent mapping only > internally a

[PATCH] btrfs: remove wrong comment for can_nocow_extent()

2021-01-27 Thread fdmanana
From: Filipe Manana The comment for can_nocow_extent() says that the function will flush ordered extents, however that never happens and was never true before the comment was added in commit e4ecaf90bc13 ("btrfs: add comments for btrfs_check_can_nocow() and can_nocow_extent()"). This is true only

Re: [PATCH v4 00/12] Improve preemptive ENOSPC flushing

2021-01-27 Thread David Sterba
On Tue, Jan 26, 2021 at 04:24:24PM -0500, Josef Bacik wrote: > v3->v4: > - Added some comments. > - Updated the tracepoint for the preemptive flushing to take a bool instead of > an int. > - Removed 'inline' from need_preemptive_flushing. Thanks, I'll add it to misc-next.

Re: [PATCH 6/7] btrfs: remove unnecessary check_parent_dirs_for_sync()

2021-01-27 Thread Josef Bacik
On 1/27/21 5:34 AM, fdman...@kernel.org wrote: From: Filipe Manana Whenever we fsync an inode, if it is a directory, a regular file that was created in the current transaction or has last_unlink_trans set to the generation of the current transaction, we check if any of its ancestor inodes (and

Re: [PATCH] btrfs: remove wrong comment for can_nocow_extent()

2021-01-27 Thread Josef Bacik
On 1/27/21 10:05 AM, fdman...@kernel.org wrote: From: Filipe Manana The comment for can_nocow_extent() says that the function will flush ordered extents, however that never happens and was never true before the comment was added in commit e4ecaf90bc13 ("btrfs: add comments for btrfs_check_can_n

Re: [PATCH v3 01/12] btrfs: make flush_space take a enum btrfs_flush_state instead of int

2021-01-27 Thread David Sterba
On Tue, Jan 26, 2021 at 03:32:36PM -0500, Josef Bacik wrote: > On 1/26/21 1:36 PM, David Sterba wrote: > > On Fri, Oct 09, 2020 at 09:28:18AM -0400, Josef Bacik wrote: > >> I got a automated message from somebody who runs clang against our > >> kernels and it's because I used the wrong enum type fo

Re: [PATCH 7/7] btrfs: make concurrent fsyncs wait less when waiting for a transaction commit

2021-01-27 Thread Josef Bacik
On 1/27/21 5:35 AM, fdman...@kernel.org wrote: From: Filipe Manana Often an fsync needs to fallback to a transaction commit for several reasons (to ensure consistency after a power failure, a new block group was allocated or a temporary error such as ENOMEM or ENOSPC happened). In that case th

Re: [PATCH] btrfs: Simplify the calculation of variables

2021-01-27 Thread Josef Bacik
On 1/27/21 3:11 AM, Abaci Team wrote: Fix the following coccicheck warnings: ./fs/btrfs/delayed-inode.c:1157:39-41: WARNING !A || A && B is equivalent to !A || B. Reported-by: Abaci Robot Suggested-by: Jiapeng Zhong Signed-off-by: Abaci Team Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH 6/7] btrfs: remove unnecessary check_parent_dirs_for_sync()

2021-01-27 Thread Filipe Manana
On Wed, Jan 27, 2021 at 3:23 PM Josef Bacik wrote: > > On 1/27/21 5:34 AM, fdman...@kernel.org wrote: > > From: Filipe Manana > > > > Whenever we fsync an inode, if it is a directory, a regular file that was > > created in the current transaction or has last_unlink_trans set to the > > generation

Re: [PATCH 6/7] btrfs: remove unnecessary check_parent_dirs_for_sync()

2021-01-27 Thread Josef Bacik
On 1/27/21 10:36 AM, Filipe Manana wrote: On Wed, Jan 27, 2021 at 3:23 PM Josef Bacik wrote: On 1/27/21 5:34 AM, fdman...@kernel.org wrote: From: Filipe Manana Whenever we fsync an inode, if it is a directory, a regular file that was created in the current transaction or has last_unlink_tra

Re: [PATCH 0/7] btrfs: more performance improvements for dbench workloads

2021-01-27 Thread Josef Bacik
On 1/27/21 5:34 AM, fdman...@kernel.org wrote: From: Filipe Manana The following patchset brings one more batch of performance improvements with dbench workloads, or anything that mixes file creation, file writes, renames, unlinks, etc with fsync like dbench does. This patchset is mostly based

Re: [PATCH v5 02/18] btrfs: set UNMAPPED bit early in btrfs_clone_extent_buffer() for subpage support

2021-01-27 Thread Josef Bacik
On 1/26/21 3:33 AM, Qu Wenruo wrote: For the incoming subpage support, UNMAPPED extent buffer will have different behavior in btrfs_release_extent_buffer(). This means we need to set UNMAPPED bit early before calling btrfs_release_extent_buffer(). Currently there is only one caller which relies

Re: [PATCH v5 04/18] btrfs: make attach_extent_buffer_page() handle subpage case

2021-01-27 Thread Josef Bacik
On 1/26/21 3:33 AM, Qu Wenruo wrote: For subpage case, we need to allocate additional memory for each metadata page. So we need to: - Allow attach_extent_buffer_page() to return int to indicate allocation failure - Allow manually pre-allocate subpage memory for alloc_extent_buffer() As w

Re: [PATCH v5 01/18] btrfs: merge PAGE_CLEAR_DIRTY and PAGE_SET_WRITEBACK to PAGE_START_WRITEBACK

2021-01-27 Thread Josef Bacik
On 1/26/21 3:33 AM, Qu Wenruo wrote: PAGE_CLEAR_DIRTY and PAGE_SET_WRITEBACK are two defines used in __process_pages_contig(), to let the function know to clear page dirty bit and then set page writeback. However page writeback and dirty bits are conflicting (at least for sector size == PAGE_SIZ

Re: [PATCH v5 00/18] btrfs: add read-only support for subpage sector size

2021-01-27 Thread Josef Bacik
On 1/26/21 3:33 AM, Qu Wenruo wrote: Patches can be fetched from github: https://github.com/adam900710/linux/tree/subpage Currently the branch also contains partial RW data support (still some ordered extent and data csum mismatch problems) Great thanks to David/Nikolay/Josef for their effort re

Re: [PATCH v5 05/18] btrfs: make grab_extent_buffer_from_page() handle subpage case

2021-01-27 Thread Josef Bacik
On 1/26/21 3:33 AM, Qu Wenruo wrote: For subpage case, grab_extent_buffer() can't really get an extent buffer just from btrfs_subpage. We have radix tree lock protecting us from inserting the same eb into the tree. Thus we don't really need to do the extra hassle, just let alloc_extent_buffer()

Re: [PATCH v5 06/18] btrfs: support subpage for extent buffer page release

2021-01-27 Thread Josef Bacik
On 1/26/21 3:33 AM, Qu Wenruo wrote: In btrfs_release_extent_buffer_pages(), we need to add extra handling for subpage. Introduce a helper, detach_extent_buffer_page(), to do different handling for regular and subpage cases. For subpage case, handle detaching page private. For unmapped (dummy

Re: [PATCH v5 07/18] btrfs: attach private to dummy extent buffer pages

2021-01-27 Thread Josef Bacik
On 1/26/21 3:33 AM, Qu Wenruo wrote: There are locations where we allocate dummy extent buffers for temporary usage, like in tree_mod_log_rewind() or get_old_root(). These dummy extent buffers will be handled by the same eb accessors, and if they don't have page::private subpage eb accessors cou

Re: [PATCH v5 08/18] btrfs: introduce helpers for subpage uptodate status

2021-01-27 Thread Josef Bacik
On 1/26/21 3:33 AM, Qu Wenruo wrote: Introduce the following functions to handle subpage uptodate status: - btrfs_subpage_set_uptodate() - btrfs_subpage_clear_uptodate() - btrfs_subpage_test_uptodate() These helpers can only be called when the page has subpage attached and the range is ens

Re: [PATCH v5 09/18] btrfs: introduce helpers for subpage error status

2021-01-27 Thread Josef Bacik
On 1/26/21 3:33 AM, Qu Wenruo wrote: Introduce the following functions to handle subpage error status: - btrfs_subpage_set_error() - btrfs_subpage_clear_error() - btrfs_subpage_test_error() These helpers can only be called when the page has subpage attached and the range is ensured to be i

Re: [PATCH v5 10/18] btrfs: support subpage in set/clear_extent_buffer_uptodate()

2021-01-27 Thread Josef Bacik
On 1/26/21 3:33 AM, Qu Wenruo wrote: To support subpage in set_extent_buffer_uptodate and clear_extent_buffer_uptodate we only need to use the subpage-aware helpers to update the page bits. Signed-off-by: Qu Wenruo Reviewed-by: David Sterba Signed-off-by: David Sterba Reviewed-by: Josef Bac

Re: [PATCH v5 11/18] btrfs: support subpage in btrfs_clone_extent_buffer

2021-01-27 Thread Josef Bacik
On 1/26/21 3:33 AM, Qu Wenruo wrote: For btrfs_clone_extent_buffer(), it's mostly the same code of __alloc_dummy_extent_buffer(), except it has extra page copy. So to make it subpage compatible, we only need to: - Call set_extent_buffer_uptodate() instead of SetPageUptodate() This will set c

Re: [PATCH v5 12/18] btrfs: support subpage in try_release_extent_buffer()

2021-01-27 Thread Josef Bacik
On 1/26/21 3:33 AM, Qu Wenruo wrote: Unlike the original try_release_extent_buffer(), try_release_subpage_extent_buffer() will iterate through all the ebs in the page, and try to release each. We can release the full page only after there's no private attached, which means all ebs of that page h

Re: [PATCH v5 13/18] btrfs: introduce read_extent_buffer_subpage()

2021-01-27 Thread Josef Bacik
On 1/26/21 3:33 AM, Qu Wenruo wrote: Introduce a helper, read_extent_buffer_subpage(), to do the subpage extent buffer read. The difference between regular and subpage routines are: - No page locking Here we completely rely on extent locking. Page locking can reduce the concurrency greatl

Re: [PATCH v5 14/18] btrfs: support subpage in endio_readpage_update_page_status()

2021-01-27 Thread Josef Bacik
On 1/26/21 3:33 AM, Qu Wenruo wrote: To handle subpage status update, add the following: - Use btrfs_page_*() subpage-aware helpers to update page status Now we can handle both cases well. - No page unlock for subpage metadata Since subpage metadata doesn't utilize page locking at all, sk

Re: [PATCH v5 15/18] btrfs: introduce subpage metadata validation check

2021-01-27 Thread Josef Bacik
On 1/26/21 3:33 AM, Qu Wenruo wrote: For subpage metadata validation check, there are some differences: - Read must finish in one bvec Since we're just reading one subpage range in one page, it should never be split into two bios nor two bvecs. - How to grab the existing eb Instead of

Re: misc bio allocation cleanups

2021-01-27 Thread Jens Axboe
On 1/26/21 7:52 AM, Christoph Hellwig wrote: > Hi Jens, > > this series contains various cleanups for how bios are allocated or > initialized plus related fallout. Applied, thanks. -- Jens Axboe

Re: [PATCH v5 16/18] btrfs: introduce btrfs_subpage for data inodes

2021-01-27 Thread Josef Bacik
On 1/26/21 3:34 AM, Qu Wenruo wrote: To support subpage sector size, data also need extra info to make sure which sectors in a page are uptodate/dirty/... This patch will make pages for data inodes to get btrfs_subpage structure attached, and detached when the page is freed. This patch also sli

Re: [PATCH v5 17/18] btrfs: integrate page status update for data read path into begin/end_page_read()

2021-01-27 Thread Josef Bacik
On 1/26/21 3:34 AM, Qu Wenruo wrote: In btrfs data page read path, the page status update are handled in two different locations: btrfs_do_read_page() { while (cur <= end) { /* No need to read from disk */ if (HOLE/PREALLOC/INLINE){

Re: [PATCH v5 18/18] btrfs: allow RO mount of 4K sector size fs on 64K page system

2021-01-27 Thread Josef Bacik
On 1/26/21 3:34 AM, Qu Wenruo wrote: This adds the basic RO mount ability for 4K sector size on 64K page system. Currently we only plan to support 4K and 64K page system. Signed-off-by: Qu Wenruo Signed-off-by: David Sterba Reviewed-by: Josef Bacik Thanks, Josef

ENOSPC in btrfs_run_delayed_refs with 5.10.8 + zstd

2021-01-27 Thread Martin Raiber
Hi, seems 5.10.8 still has the ENOSPC issue when compression is used (compress-force=zstd,space_cache=v2): Jan 27 11:02:14  kernel: [248571.569840] [ cut here ] Jan 27 11:02:14  kernel: [248571.569843] BTRFS: Transaction aborted (error -28) Jan 27 11:02:14  kernel: [24857

Re: [PATCH v14 12/42] btrfs: calculate allocation offset for conventional zones

2021-01-27 Thread Josef Bacik
On 1/25/21 9:24 PM, Naohiro Aota wrote: Conventional zones do not have a write pointer, so we cannot use it to determine the allocation offset if a block group contains a conventional zone. But instead, we can consider the end of the last allocated extent in the block group as an allocation offs

Re: [PATCH v14 13/42] btrfs: track unusable bytes for zones

2021-01-27 Thread Josef Bacik
On 1/25/21 9:24 PM, Naohiro Aota wrote: In zoned btrfs a region that was once written then freed is not usable until we reset the underlying zone. So we need to distinguish such unusable space from usable free space. Therefore we need to introduce the "zone_unusable" field to the block group st

Re: [PATCH v14 41/42] btrfs: serialize log transaction on ZONED mode

2021-01-27 Thread Josef Bacik
On 1/25/21 9:25 PM, Naohiro Aota wrote: This is the 2/3 patch to enable tree-log on ZONED mode. Since we can start more than one log transactions per subvolume simultaneously, nodes from multiple transactions can be allocated interleaved. Such mixed allocation results in non-sequential writes at

Re: [PATCH v14 22/42] btrfs: split ordered extent when bio is sent

2021-01-27 Thread Josef Bacik
On 1/25/21 9:25 PM, Naohiro Aota wrote: For a zone append write, the device decides the location the data is written to. Therefore we cannot ensure that two bios are written consecutively on the device. In order to ensure that a ordered extent maps to a contiguous region on disk, we need to maint

Re: [PATCH] btrfs: remove wrong comment for can_nocow_extent()

2021-01-27 Thread David Sterba
On Wed, Jan 27, 2021 at 03:05:41PM +, fdman...@kernel.org wrote: > From: Filipe Manana > > The comment for can_nocow_extent() says that the function will flush > ordered extents, however that never happens and was never true before the > comment was added in commit e4ecaf90bc13 ("btrfs: add c

Re: [PATCH v4 04/18] btrfs: make attach_extent_buffer_page() to handle subpage case

2021-01-27 Thread David Sterba
On Tue, Jan 26, 2021 at 03:29:17PM +0800, Qu Wenruo wrote: > On 2021/1/20 上午6:35, David Sterba wrote: > > On Tue, Jan 19, 2021 at 04:54:28PM -0500, Josef Bacik wrote: > >> On 1/16/21 2:15 AM, Qu Wenruo wrote: > >>> +/* For rare cases where we need to pre-allocate a btrfs_subpage > >>> structure */

Re: btrfs becomes read-only

2021-01-27 Thread Chris Murphy
On Wed, Jan 27, 2021 at 1:57 AM Alexey Isaev wrote: > > kernel version: > > aleksey@host:~$ sudo uname --all > Linux host 4.15.0-132-generic #136~16.04.1-Ubuntu SMP Tue Jan 12 > 18:22:20 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux This is an old and EOL kernel. It could be a long fixed Btrfs bug that

Re: btrfs becomes read-only

2021-01-27 Thread Chris Murphy
On Wed, Jan 27, 2021 at 6:05 AM Alexey Isaev wrote: > > I managed to run btrs check, but it didn't solve the problem: > > aleksey@host:~$ sudo btrfs check --repair /dev/sdg OK it's risky to run --repair without a developer giving a go ahead, in particular with older versions of btrfs-progs. There

Re: Only one subvolume can be mounted after replace/balance

2021-01-27 Thread Chris Murphy
On Wed, Jan 27, 2021 at 6:10 AM Jakob Schöttl wrote: > > Thank you Chris, it's resolved now, see below. > > Am 25.01.21 um 23:47 schrieb Chris Murphy: > > On Sat, Jan 23, 2021 at 7:50 AM Jakob Schöttl wrote: > >> Hi, > >> > >> In short: > >> When mounting a second subvolume from a pool, I get thi

Re: ENOSPC in btrfs_run_delayed_refs with 5.10.8 + zstd

2021-01-27 Thread Chris Murphy
On Wed, Jan 27, 2021 at 10:27 AM Martin Raiber wrote: > > Hi, > > seems 5.10.8 still has the ENOSPC issue when compression is used > (compress-force=zstd,space_cache=v2): > > Jan 27 11:02:14 kernel: [248571.569840] [ cut here ] > Jan 27 11:02:14 kernel: [248571.569843] B

Re: [PATCH v5 00/18] btrfs: add read-only support for subpage sector size

2021-01-27 Thread Qu Wenruo
On 2021/1/28 上午12:17, Josef Bacik wrote: On 1/26/21 3:33 AM, Qu Wenruo wrote: Patches can be fetched from github: https://github.com/adam900710/linux/tree/subpage Currently the branch also contains partial RW data support (still some ordered extent and data csum mismatch problems) Great than