[PATCH 02/35] fscrypt: add per-extent encryption support

2023-09-26 Thread Josef Bacik
s we're deriving a per-extent key to use for the encryption, the inode still controls the policy and access to the master key. Signed-off-by: Josef Bacik --- fs/crypto/crypto.c | 10 ++- fs/crypto/fscrypt_private.h | 45 +++ fs/crypto/inline_crypt.c| 84

[PATCH 01/35] fscrypt: rename fscrypt_info => fscrypt_inode_info

2023-09-26 Thread Josef Bacik
We are going to track per-extent information, so it'll be necessary to distinguish between inode infos and extent infos. Rename fscrypt_info to fscrypt_inode_info, adjusting any lines that now exceed 80 characters. Signed-off-by: Josef Bacik --- fs/crypto/crypto.c

[PATCH 00/35] btrfs: add fscrypt support

2023-09-26 Thread Josef Bacik
https://github.com/kdave/btrfs-devel/tree/for-next [4] https://github.com/josefbacik/linux/tree/fscrypt Josef Bacik (20): fscrypt: rename fscrypt_info => fscrypt_inode_info fscrypt: add per-extent encryption support fscrypt: disable all but standard v2 policies for extent encryption blk-crypto

Re: [RFC PATCH 0/4] fscrypt: add support for per-extent encryption

2023-09-15 Thread Josef Bacik
On Thu, Sep 14, 2023 at 11:48:16PM -0700, Eric Biggers wrote: > On Thu, Sep 14, 2023 at 08:47:41PM -0400, Josef Bacik wrote: > > Hello, > > > > This is meant as a replacement for the last set of patches Sweet Tea sent > > [1]. > > This is an attempt to find a di

[PATCH 3/4] fscrypt: disable all but standard v2 policies for extent encryption

2023-09-14 Thread Josef Bacik
The different encryption related options for fscrypt are too numerous to support for extent based encryption. Support for a few of these options could possibly be added, but since they're niche options simply reject them for file systems using extent based encryption. Signed-off-by: Josef

[PATCH 4/4] blk-crypto: add a process bio callback

2023-09-14 Thread Josef Bacik
ative encryption if this callback is set. Signed-off-by: Josef Bacik --- block/blk-crypto-fallback.c | 18 ++ block/blk-crypto-profile.c | 2 ++ block/blk-crypto.c | 6 +- fs/crypto/inline_crypt.c| 3 ++- include/linux/blk-crypto.h | 9 +++-- include/linux/fscr

[PATCH 2/4] fscrypt: add per-extent encryption support

2023-09-14 Thread Josef Bacik
s we're deriving a per-extent key to use for the encryption, the inode still controls the policy and access to the master key. Signed-off-by: Josef Bacik --- fs/crypto/crypto.c | 10 ++- fs/crypto/fscrypt_private.h | 36 ++ fs/crypto/inline_crypt.c| 34

[PATCH 1/4] fscrypt: rename fscrypt_info => fscrypt_inode_info

2023-09-14 Thread Josef Bacik
We are going to track per-extent information, so it'll be necessary to distinguish between inode infos and extent infos. Rename fscrypt_info to fscrypt_inode_info, adjusting any lines that now exceed 80 characters. Signed-off-by: Josef Bacik --- fs/crypto/crypto.c

[RFC PATCH 0/4] fscrypt: add support for per-extent encryption

2023-09-14 Thread Josef Bacik
ver.1693630890.git.sweettea-ker...@dorminy.me/ Josef Bacik (4): fscrypt: rename fscrypt_info => fscrypt_inode_info fscrypt: add per-extent encryption support fscrypt: disable all but standard v2 policies for extent encryption blk-crypto: add a process bio callback block/blk-crypto-f

Re: [PATCH 21/42] btrfs: make process_one_page() to handle subpage locking

2021-04-16 Thread Josef Bacik
ed by process_one_page() to keep btrfs_subpage::writers consistent. This will be handled in next patch. Signed-off-by: Qu Wenruo Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH 20/42] btrfs: make end_bio_extent_writepage() to be subpage compatible

2021-04-16 Thread Josef Bacik
On 4/15/21 1:04 AM, Qu Wenruo wrote: Now in end_bio_extent_writepage(), the only subpage incompatible code is the end_page_writeback(). Just call the subpage helpers. Signed-off-by: Qu Wenruo Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH 19/42] btrfs: make __process_pages_contig() to handle subpage dirty/error/writeback status

2021-04-16 Thread Josef Bacik
On 4/15/21 1:04 AM, Qu Wenruo wrote: For __process_pages_contig() and process_one_page(), to handle subpage we only need to pass bytenr in and call subpage helpers to handle dirty/error/writeback status. Signed-off-by: Qu Wenruo Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH 18/42] btrfs: make btrfs_dirty_pages() to be subpage compatible

2021-04-16 Thread Josef Bacik
On 4/15/21 1:04 AM, Qu Wenruo wrote: Since the extent io tree operations in btrfs_dirty_pages() are already subpage compatible, we only need to make the page status update to use subpage helpers. Signed-off-by: Qu Wenruo Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH 17/42] btrfs: only require sector size alignment for end_bio_extent_writepage()

2021-04-16 Thread Josef Bacik
On 4/15/21 1:04 AM, Qu Wenruo wrote: Just like read page, for subpage support we only require sector size alignment. So change the error message condition to only require sector alignment. This should not affect existing code, as for regular sectorsize == PAGE_SIZE case, we are still requiring

Re: [PATCH 16/42] btrfs: provide btrfs_page_clamp_*() helpers

2021-04-16 Thread Josef Bacik
*() calls can be converted to use btrfs_page_clamp_*() directly, but that would make us to do unnecessary clamp operations. Signed-off-by: Qu Wenruo Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH 15/42] btrfs: refactor the page status update into process_one_page()

2021-04-16 Thread Josef Bacik
have questions about the pages_processed thing, but that can be addressed in the other patch and then will trickle down to here, so you can add Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH 14/42] btrfs: pass bytenr directly to __process_pages_contig()

2021-04-16 Thread Josef Bacik
On 4/15/21 1:04 AM, Qu Wenruo wrote: As a preparation for incoming subpage support, we need bytenr passed to __process_pages_contig() directly, not the current page index. So change the parameter and all callers to pass bytenr in. With the modification, here we need to replace the old @index_re

Re: [PATCH 13/42] btrfs: rename PagePrivate2 to PageOrdered inside btrfs

2021-04-16 Thread Josef Bacik
bit added, so reader who is still uncertain about the page Ordered status, will find the comment pretty easily. Signed-off-by: Qu Wenruo Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH 12/42] btrfs: make Private2 lifespan more consistent

2021-04-16 Thread Josef Bacik
, just meaning the page has unfinished ordered extent attached to it. Signed-off-by: Qu Wenruo Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH 11/42] btrfs: refactor btrfs_invalidatepage()

2021-04-16 Thread Josef Bacik
On 4/15/21 1:04 AM, Qu Wenruo wrote: This patch will refactor btrfs_invalidatepage() for the incoming subpage support. The invovled modifcations are: - Use while() loop instead of "goto again;" - Use single variable to determine whether to delete extent states Each branch will also have comme

Re: [PATCH 10/42] btrfs: update the comments in btrfs_invalidatepage()

2021-04-16 Thread Josef Bacik
t_on_page_writeback() and Private2 to prevent double accounting on ordered extent. Signed-off-by: Qu Wenruo Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH 09/42] btrfs: refactor how we finish ordered extent io for endio functions

2021-04-16 Thread Josef Bacik
On 4/15/21 1:04 AM, Qu Wenruo wrote: Btrfs has two endio functions to mark certain io range finished for ordered extents: - __endio_write_update_ordered() This is for direct IO - btrfs_writepage_endio_finish_ordered() This for buffered IO. However they go different routines to handle orde

Re: [PATCH 08/42] btrfs: pass btrfs_inode into btrfs_writepage_endio_finish_ordered()

2021-04-16 Thread Josef Bacik
On 4/15/21 1:04 AM, Qu Wenruo wrote: There is a pretty bad abuse of btrfs_writepage_endio_finish_ordered() in end_compressed_bio_write(). It passes compressed pages to btrfs_writepage_endio_finish_ordered(), which is only supposed to accept inode pages. Thankfully the important info here is the

Re: [PATCH 07/42] btrfs: use u32 for length related members of btrfs_ordered_extent

2021-04-16 Thread Josef Bacik
On 4/15/21 1:04 AM, Qu Wenruo wrote: Unlike btrfs_file_extent_item, btrfs_ordered_extent has its length limit (BTRFS_MAX_EXTENT_SIZE), which is far smaller than U32_MAX. Using u64 for those length related members are just a waste of memory. This patch will make the following members u32: - num_

Re: [PATCH 06/42] btrfs: allow btrfs_bio_fits_in_stripe() to accept bio without any page

2021-04-16 Thread Josef Bacik
against stripe boundary, even adding the range to an empty bio. This tiny refactor is for the incoming change, but on its own, regular sectorsize == PAGE_SIZE is not affected anyway. Signed-off-by: Qu Wenruo Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH 05/42] btrfs: remove the unused parameter @len for btrfs_bio_fits_in_stripe()

2021-04-16 Thread Josef Bacik
On 4/15/21 1:04 AM, Qu Wenruo wrote: The parameter @len is not really used in btrfs_bio_fits_in_stripe(), just remove it. Signed-off-by: Qu Wenruo Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH 02/42] btrfs: introduce write_one_subpage_eb() function

2021-04-16 Thread Josef Bacik
On 4/15/21 7:25 PM, Qu Wenruo wrote: On 2021/4/16 上午3:03, Josef Bacik wrote: On 4/15/21 1:04 AM, Qu Wenruo wrote: The new function, write_one_subpage_eb(), as a subroutine for subpage metadata write, will handle the extent buffer bio submission. The major differences between the new

Re: [PATCH 04/42] btrfs: introduce submit_eb_subpage() to submit a subpage metadata page

2021-04-16 Thread Josef Bacik
On 4/15/21 7:28 PM, Qu Wenruo wrote: On 2021/4/16 上午3:27, Josef Bacik wrote: On 4/15/21 1:04 AM, Qu Wenruo wrote: The new function, submit_eb_subpage(), will submit all the dirty extent buffers in the page. The major difference between submit_eb_page() and submit_eb_subpage() is: - How to

Re: [PATCH 04/42] btrfs: introduce submit_eb_subpage() to submit a subpage metadata page

2021-04-15 Thread Josef Bacik
On 4/15/21 1:04 AM, Qu Wenruo wrote: The new function, submit_eb_subpage(), will submit all the dirty extent buffers in the page. The major difference between submit_eb_page() and submit_eb_subpage() is: - How to grab extent buffer Now we use find_extent_buffer_nospinlock() other than using

Re: [PATCH 03/42] btrfs: make lock_extent_buffer_for_io() to be subpage compatible

2021-04-15 Thread Josef Bacik
On 4/15/21 1:04 AM, Qu Wenruo wrote: For subpage metadata, we don't use page locking at all. So just skip the page locking part for subpage. All the remaining routine can be reused. Signed-off-by: Qu Wenruo Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH 02/42] btrfs: introduce write_one_subpage_eb() function

2021-04-15 Thread Josef Bacik
On 4/15/21 1:04 AM, Qu Wenruo wrote: The new function, write_one_subpage_eb(), as a subroutine for subpage metadata write, will handle the extent buffer bio submission. The major differences between the new write_one_subpage_eb() and write_one_eb() is: - No page locking When entering write_on

Re: [PATCH 01/42] btrfs: introduce end_bio_subpage_eb_writepage() function

2021-04-15 Thread Josef Bacik
On 4/15/21 1:04 AM, Qu Wenruo wrote: The new function, end_bio_subpage_eb_writepage(), will handle the metadata writeback endio. The major differences involved are: - How to grab extent buffer Now page::private is a pointer to btrfs_subpage, we can no longer grab extent buffer directly.

Re: [PATCH v4 3/3] btrfs: zoned: automatically reclaim zones

2021-04-15 Thread Josef Bacik
On 4/15/21 9:58 AM, Johannes Thumshirn wrote: When a file gets deleted on a zoned file system, the space freed is not returned back into the block group's free space, but is migrated to zone_unusable. As this zone_unusable space is behind the current write pointer it is not possible to use it fo

Re: [PATCH v4 2/3] btrfs: rename delete_unused_bgs_mutex

2021-04-15 Thread Josef Bacik
On 4/15/21 9:58 AM, Johannes Thumshirn wrote: As a preparation for another user, rename the unused_bgs_mutex into reclaim_bgs_lock. Signed-off-by: Johannes Thumshirn Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH v4 1/3] btrfs: zoned: reset zones of relocated block groups

2021-04-15 Thread Josef Bacik
ld disable discard per bg so we don't discard at all during the relocation, and then discard the whole block group no matter if we have zoned or not. However not really something you need to do, just thinking out loud Reviewed-by: Josef Bacik Thanks, Josef

Re: [RFC v3 0/2] vfs / btrfs: add support for ustat()

2021-04-15 Thread Josef Bacik
On 4/15/21 1:53 PM, Luis Chamberlain wrote: On Wed, Aug 23, 2017 at 3:31 PM Jeff Mahoney wrote: On 8/15/14 5:29 AM, Al Viro wrote: On Thu, Aug 14, 2014 at 07:58:56PM -0700, Luis R. Rodriguez wrote: Christoph had noted that this seemed associated to the problem that the btrfs uses different

Re: [PATCH] btrfs: fix race between transaction aborts and fsyncs leading to use-after-free

2021-04-08 Thread Josef Bacik
andles. This makes the transaction kthread wait for all the tasks attached to the transaction to be done with the transaction before dropping the log roots and doing other cleanups. Fixes: ef67963dac255b ("btrfs: drop logs when we've aborted a transaction") Signed-off-by: Filipe Manana Nice catch, Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH] btrfs-progs: Fix null pointer deref in balance_level

2021-04-08 Thread Josef Bacik
ht->len in "blocksize". Fixes #296 Signed-off-by: Nikolay Borisov Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH] btrfs: return whole extents in fiemap

2021-04-08 Thread Josef Bacik
ercised by xfstest generic/483, and I have written a new xfstest for checking we don't have backwards or zero-length fiemaps for cases like those above. Signed-off-by: Boris Burkov Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH v2] btrfs: do more graceful error/warning for 32bit kernel

2021-04-08 Thread Josef Bacik
enruo This doesn't apply cleanly to misc-next so it needs to be respun, but otherwise Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH] btrfs: Correct try_lock_extent() usage in read_extent_buffer_subpage()

2021-04-08 Thread Josef Bacik
locking the extent. Signed-off-by: Goldwyn Rodrigues Reviewed-by: Josef Bacik Thanks, Josef

Re: ENOSPC in btrfs_run_delayed_refs with 5.10.8

2021-04-08 Thread Josef Bacik
On 4/8/21 12:10 PM, Martin Raiber wrote: On 11.03.2021 18:58 Martin Raiber wrote: On 01.02.2021 23:08 Martin Raiber wrote: On 27.01.2021 22:03 Chris Murphy wrote: On Wed, Jan 27, 2021 at 10:27 AM Martin Raiber wrote: Hi, seems 5.10.8 still has the ENOSPC issue when compression is used (com

Re: [PATCH v2] btrfs: zoned: move superblock logging zone location

2021-04-08 Thread Josef Bacik
reserved to avoid confusion. Signed-off-by: Naohiro Aota Thanks Naohiro, this makes it much easier to understand, Reviewed-by: Josef Bacik Dave, I know you're on vacation, this needs to go to Linus for this cycle so we don't have two different SB slots for two different kernels. I don

Re: [PATCH] btrfs: zoned: move superblock logging zone location

2021-04-07 Thread Josef Bacik
On 4/7/21 2:31 PM, Johannes Thumshirn wrote: On 07/04/2021 19:54, Josef Bacik wrote: On 3/15/21 1:53 AM, Naohiro Aota wrote: This commit moves the location of superblock logging zones. The location of the logging zones are determined based on fixed block addresses instead of on fixed zone

Re: [PATCH] btrfs: zoned: move superblock logging zone location

2021-04-07 Thread Josef Bacik
On 3/15/21 1:53 AM, Naohiro Aota wrote: This commit moves the location of superblock logging zones. The location of the logging zones are determined based on fixed block addresses instead of on fixed zone numbers. By locating the superblock zones using fixed addresses, we can scan a dumped file

Re: Aw: Re: Re: Help needed with filesystem errors: parent transid verify failed

2021-03-29 Thread Josef Bacik
On 3/29/21 4:42 AM, B A wrote: Gesendet: Montag, 29. März 2021 um 08:09 Uhr Von: "Chris Murphy" An: "B A" , "Btrfs BTRFS" Cc: "Qu Wenruo" , "Josef Bacik" Betreff: Re: Re: Help needed with filesystem errors: parent transid verify failed […]

[PATCH] btrfs: use percpu_read_positive instead of sum_positive for need_preempt

2021-03-24 Thread Josef Bacik
baseline currentdiff == elapsed 47.41 47.40 -0.03% Signed-off-by: Josef Bacik --- fs/btrfs/space-info.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/space-info.c b/fs/btrfs/space-info.c index

Re: [PATCH v8 00/10] fs: interface for directly reading/writing compressed data

2021-03-19 Thread Josef Bacik
On 3/19/21 4:08 PM, Linus Torvalds wrote: On Fri, Mar 19, 2021 at 11:21 AM Josef Bacik wrote: Can we get some movement on this? Omar is sort of spinning his wheels here trying to get this stuff merged, no major changes have been done in a few postings. I'm not Al, and I absolutely d

Re: [PATCH v8 00/10] fs: interface for directly reading/writing compressed data

2021-03-19 Thread Josef Bacik
On 3/16/21 3:42 PM, Omar Sandoval wrote: From: Omar Sandoval This series adds an API for reading compressed data on a filesystem without decompressing it as well as support for writing compressed data directly to the filesystem. As with the previous submissions, I've included a man page patch d

Re: [PATCH v2 2/2] btrfs: zoned: automatically reclaim zones

2021-03-19 Thread Josef Bacik
On 3/19/21 6:48 AM, Johannes Thumshirn wrote: When a file gets deleted on a zoned file system, the space freed is not returned back into the block group's free space, but is migrated to zone_unusable. As this zone_unusable space is behind the current write pointer it is not possible to use it fo

Re: [PATCH v2 1/2] btrfs: rename delete_unused_bgs_mutex

2021-03-19 Thread Josef Bacik
On 3/19/21 6:48 AM, Johannes Thumshirn wrote: As a preparation for another user, rename the unused_bgs_mutex into reclaim_bgs_lock. Signed-off-by: Johannes Thumshirn Reviewed-by: Josef Bacik Thanks, Josef

Re: [PATCH 0/3] Handle bad dev_root properly with rescue=all

2021-03-18 Thread Josef Bacik
On 3/18/21 11:43 AM, David Sterba wrote: On Thu, Mar 11, 2021 at 11:23:13AM -0500, Josef Bacik wrote: [...] rescue=all working without panicing the machine, and I verified everything by using btrfs-corrupt-block to corrupt a dev root of a file system. Thanks, We need to have such testing

Re: [PATCH] btrfs: zoned: automatically reclaim zones

2021-03-17 Thread Josef Bacik
On 3/17/21 6:38 AM, Johannes Thumshirn wrote: When a file gets deleted on a zoned file system, the space freed is no returned back into the block group's free space, but is migrated to zone_unusable. As this zone_unusable space is behind the current write pointer it is not possible to use it for

Re: [PATCH 0/3] Handle bad dev_root properly with rescue=all

2021-03-17 Thread Josef Bacik
On 3/17/21 8:27 AM, David Sterba wrote: On Thu, Mar 11, 2021 at 11:23:13AM -0500, Josef Bacik wrote: Hello, My recent debugging session with Neal's broken filesystem uncovered a glaring hole in my rescue=all patches, they don't deal with a NULL dev_root properly. In testing I only e

[PATCH v8 39/39] btrfs: check return value of btrfs_commit_transaction in relocation

2021-03-12 Thread Josef Bacik
There's a few places where we don't check the return value of btrfs_commit_transaction in relocation.c. Thankfully all these places have straightforward error handling, so simply change all of the sites at once. Reviewed-by: Qu Wenruo Signed-off-by: Josef Bacik --- fs/btrfs/reloca

[PATCH v8 38/39] btrfs: do proper error handling in merge_reloc_roots

2021-03-12 Thread Josef Bacik
ition properly, ASSERT()'ing for developers. Signed-off-by: Josef Bacik --- fs/btrfs/relocation.c | 25 +++-- 1 file changed, 23 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c index 621c4284d158..557d4f09ce7f 100644 ---

[PATCH v8 37/39] btrfs: handle extent corruption with select_one_root properly

2021-03-12 Thread Josef Bacik
In corruption cases we could have paths from a block up to no root at all, and thus we'll BUG_ON(!root) in select_one_root. Handle this by adding an ASSERT() for developers, and returning an error for normal users. Signed-off-by: Josef Bacik --- fs/btrfs/relocation.c

[PATCH v8 36/39] btrfs: cleanup error handling in prepare_to_merge

2021-03-12 Thread Josef Bacik
Signed-off-by: Josef Bacik --- fs/btrfs/relocation.c | 10 -- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c index 857da684d415..8f7760f8fcc3 100644 --- a/fs/btrfs/relocation.c +++ b/fs/btrfs/relocation.c @@ -1880,8

[PATCH v8 34/39] btrfs: handle __add_reloc_root failures in btrfs_recover_relocation

2021-03-12 Thread Josef Bacik
We can already handle errors appropriately from this function, deal with an error coming from __add_reloc_root appropriately. Reviewed-by: Qu Wenruo Signed-off-by: Josef Bacik --- fs/btrfs/relocation.c | 12 ++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/fs/btrfs

[PATCH v8 32/39] btrfs: remove the extent item sanity checks in relocate_block_group

2021-03-12 Thread Josef Bacik
These checks are all taken care of for us by the tree checker code. Reviewed-by: Qu Wenruo Signed-off-by: Josef Bacik --- fs/btrfs/relocation.c | 29 + 1 file changed, 1 insertion(+), 28 deletions(-) diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c index

[PATCH v8 30/39] btrfs: handle extent reference errors in do_relocation

2021-03-12 Thread Josef Bacik
We can already deal with errors appropriately from do_relocation, simply handle any errors that come from changing the refs at this point cleanly. We have to abort the transaction if we fail here as we've modified metadata at this point. Signed-off-by: Josef Bacik --- fs/btrfs/relocation.

[PATCH v8 33/39] btrfs: do proper error handling in create_reloc_inode

2021-03-12 Thread Josef Bacik
s means we can avoid a transaction abort in this particular error case. Signed-off-by: Josef Bacik --- fs/btrfs/relocation.c | 39 +-- 1 file changed, 37 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c index 28

[PATCH v8 31/39] btrfs: check for BTRFS_BLOCK_FLAG_FULL_BACKREF being set improperly

2021-03-12 Thread Josef Bacik
We need to validate that a data extent item does not have the FULL_BACKREF flag set on it's flags. Reviewed-by: Qu Wenruo Signed-off-by: Josef Bacik --- fs/btrfs/tree-checker.c | 5 + 1 file changed, 5 insertions(+) diff --git a/fs/btrfs/tree-checker.c b/fs/btrfs/tree-checker.c

[PATCH v8 35/39] btrfs: do not panic in __add_reloc_root

2021-03-12 Thread Josef Bacik
opers, otherwise normal error handling will happen for normal users. Signed-off-by: Josef Bacik --- fs/btrfs/relocation.c | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c index 2fc04f96294e..857da684d415 100644 --- a/fs/btrfs/relocat

[PATCH v8 28/39] btrfs: handle btrfs_search_slot failure in replace_path

2021-03-12 Thread Josef Bacik
This can fail for any number of reasons, why bring the whole box down with it? Reviewed-by: Qu Wenruo Signed-off-by: Josef Bacik --- fs/btrfs/relocation.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c index 592b2d156626

[PATCH v8 25/39] btrfs: do proper error handling in btrfs_update_reloc_root

2021-03-12 Thread Josef Bacik
We call btrfs_update_root in btrfs_update_reloc_root, which can fail for all sorts of reasons, including IO errors. Instead of panicing the box lets return the error, now that all callers properly handle those errors. Reviewed-by: Qu Wenruo Signed-off-by: Josef Bacik --- fs/btrfs/relocation.c

[PATCH v8 24/39] btrfs: handle btrfs_update_reloc_root failure in prepare_to_merge

2021-03-12 Thread Josef Bacik
btrfs_update_reloc_root will will return errors in the future, so handle an error properly in prepare_to_merge. Signed-off-by: Josef Bacik --- fs/btrfs/relocation.c | 13 - 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c

[PATCH v8 29/39] btrfs: handle errors in reference count manipulation in replace_path

2021-03-12 Thread Josef Bacik
If any of the reference count manipulation stuff fails in replace_path we need to abort the transaction, as we've modified the blocks already. We can simply break at this point and everything will be cleaned up. Reviewed-by: Qu Wenruo Signed-off-by: Josef Bacik --- fs/btrfs/relocation.c

[PATCH v8 26/39] btrfs: convert logic BUG_ON()'s in replace_path to ASSERT()'s

2021-03-12 Thread Josef Bacik
A few BUG_ON()'s in replace_path are purely to keep us from making logical mistakes, so replace them with ASSERT()'s. Reviewed-by: Qu Wenruo Signed-off-by: Josef Bacik --- fs/btrfs/relocation.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/relocat

[PATCH v8 20/39] btrfs: validate ->reloc_root after recording root in trans

2021-03-12 Thread Josef Bacik
as the transid has already been update. Handle this case by making sure we have a root->reloc_root set after a btrfs_record_root_in_transaction call so we don't end up deref'ing a NULL pointer. Reported-by: Zygo Blaxell Signed-off-by: Josef Bacik --- fs/btrfs/relocation.c | 15

[PATCH v8 21/39] btrfs: handle btrfs_update_reloc_root failure in commit_fs_roots

2021-03-12 Thread Josef Bacik
btrfs_update_reloc_root will will return errors in the future, so handle the error properly in commit_fs_roots. Reviewed-by: Qu Wenruo Signed-off-by: Josef Bacik --- fs/btrfs/transaction.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/transaction.c b/fs/btrfs

[PATCH v8 19/39] btrfs: do proper error handling in create_reloc_root

2021-03-12 Thread Josef Bacik
We do memory allocations here, read blocks from disk, all sorts of operations that could easily fail at any given point. Instead of panicing the box, simply return the error back up the chain, all callers at this point have proper error handling. Signed-off-by: Josef Bacik --- fs/btrfs

[PATCH v8 27/39] btrfs: handle btrfs_cow_block errors in replace_path

2021-03-12 Thread Josef Bacik
If we error out cow'ing the root node when doing a replace_path then we simply unlock and free the buffer and return the error. Reviewed-by: Qu Wenruo Signed-off-by: Josef Bacik --- fs/btrfs/relocation.c | 12 ++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/fs/

[PATCH v8 22/39] btrfs: change insert_dirty_subvol to return errors

2021-03-12 Thread Josef Bacik
This will be able to return errors in the future, so change it to return an error and handle the error appropriately. Signed-off-by: Josef Bacik --- fs/btrfs/relocation.c | 14 +- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/relocation.c b/fs/btrfs

[PATCH v8 23/39] btrfs: handle btrfs_update_reloc_root failure in insert_dirty_subvol

2021-03-12 Thread Josef Bacik
btrfs_update_reloc_root will will return errors in the future, so handle the error properly in insert_dirty_subvol. Signed-off-by: Josef Bacik --- fs/btrfs/relocation.c | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c index

[PATCH v8 18/39] btrfs: have proper error handling in btrfs_init_reloc_root

2021-03-12 Thread Josef Bacik
create_reloc_root will return errors in the future, and __add_reloc_root can return -ENOMEM or -EEXIST, so handle these errors properly. Signed-off-by: Josef Bacik --- fs/btrfs/relocation.c | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/relocation.c b/fs

[PATCH v8 14/39] btrfs: handle record_root_in_trans failure in qgroup_account_snapshot

2021-03-12 Thread Josef Bacik
record_root_in_trans can fail currently, so handle this failure properly. Reviewed-by: Qu Wenruo Signed-off-by: Josef Bacik --- fs/btrfs/transaction.c | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c index 3a3a582063b4

[PATCH v8 15/39] btrfs: handle record_root_in_trans failure in btrfs_record_root_in_trans

2021-03-12 Thread Josef Bacik
record_root_in_trans can fail currently, handle this failure properly. Signed-off-by: Josef Bacik --- fs/btrfs/transaction.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c index 26c91c4eba89..7ee0199fbb95 100644 --- a/fs

[PATCH v8 16/39] btrfs: handle record_root_in_trans failure in create_pending_snapshot

2021-03-12 Thread Josef Bacik
record_root_in_trans can currently fail, so handle this failure properly. Reviewed-by: Qu Wenruo Signed-off-by: Josef Bacik --- fs/btrfs/transaction.c | 11 --- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c index

[PATCH v8 17/39] btrfs: return an error from btrfs_record_root_in_trans

2021-03-12 Thread Josef Bacik
: Johannes Thumshirn Signed-off-by: Josef Bacik --- fs/btrfs/transaction.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c index 93464b93ee9d..c24ffbf0fb74 100644 --- a/fs/btrfs/transaction.c +++ b/fs/btrfs/transaction.c @@ -408,6

[PATCH v8 13/39] btrfs: handle btrfs_record_root_in_trans failure in start_transaction

2021-03-12 Thread Josef Bacik
btrfs_record_root_in_trans will return errors in the future, so handle the error properly in start_transaction. Reviewed-by: Qu Wenruo Signed-off-by: Josef Bacik --- fs/btrfs/transaction.c | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/transaction.c b/fs

[PATCH v8 12/39] btrfs: btrfs: handle btrfs_record_root_in_trans failure in relocate_tree_block

2021-03-12 Thread Josef Bacik
btrfs_record_root_in_trans will return errors in the future, so handle the error properly in relocate_tree_block. Reviewed-by: Qu Wenruo Signed-off-by: Josef Bacik --- fs/btrfs/relocation.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/relocation.c b/fs/btrfs

[PATCH v8 11/39] btrfs: handle btrfs_record_root_in_trans failure in create_subvol

2021-03-12 Thread Josef Bacik
btrfs_record_root_in_trans will return errors in the future, so handle the error properly in create_subvol. Signed-off-by: Josef Bacik --- fs/btrfs/ioctl.c | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index e8d53fea4c61

[PATCH v8 09/39] btrfs: handle btrfs_record_root_in_trans failure in btrfs_delete_subvolume

2021-03-12 Thread Josef Bacik
btrfs_record_root_in_trans will return errors in the future, so handle the error properly in btrfs_delete_subvolume. Reviewed-by: Qu Wenruo Signed-off-by: Josef Bacik --- fs/btrfs/inode.c | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs

[PATCH v8 06/39] btrfs: do proper error handling in record_reloc_root_in_trans

2021-03-12 Thread Josef Bacik
en we look it up and see that it doesn't exist. Convert these BUG_ON()'s into ASSERT()'s + proper error handling for the case of corruption. Signed-off-by: Josef Bacik --- fs/btrfs/relocation.c | 23 +-- 1 file changed, 21 insertions(+), 2 deletions(

[PATCH v8 10/39] btrfs: handle btrfs_record_root_in_trans failure in btrfs_recover_log_trees

2021-03-12 Thread Josef Bacik
. Reviewed-by: Qu Wenruo Signed-off-by: Josef Bacik --- fs/btrfs/tree-log.c | 8 ++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/tree-log.c b/fs/btrfs/tree-log.c index 2f1acc9aea9e..941b4dd96dd0 100644 --- a/fs/btrfs/tree-log.c +++ b/fs/btrfs/tree-log.c @@ -6278,8

[PATCH v8 08/39] btrfs: handle btrfs_record_root_in_trans failure in btrfs_rename

2021-03-12 Thread Josef Bacik
btrfs_record_root_in_trans will return errors in the future, so handle the error properly in btrfs_rename. Signed-off-by: Josef Bacik --- fs/btrfs/inode.c | 7 +-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index a9fde9a22fff

[PATCH v8 05/39] btrfs: check record_root_in_trans related failures in select_reloc_root

2021-03-12 Thread Josef Bacik
We will record the fs root or the reloc root in the trans in select_reloc_root. These will actually return errors in the following patches, so check their return value here and return it up the stack. Reviewed-by: Qu Wenruo Signed-off-by: Josef Bacik --- fs/btrfs/relocation.c | 9 +++-- 1

[PATCH v8 07/39] btrfs: handle btrfs_record_root_in_trans failure in btrfs_rename_exchange

2021-03-12 Thread Josef Bacik
btrfs_record_root_in_trans will return errors in the future, so handle the error properly in btrfs_rename_exchange. Reviewed-by: Qu Wenruo Signed-off-by: Josef Bacik --- fs/btrfs/inode.c | 7 +-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs

[PATCH v8 02/39] btrfs: convert BUG_ON()'s in relocate_tree_block

2021-03-12 Thread Josef Bacik
d-by: Qu Wenruo Signed-off-by: Josef Bacik --- fs/btrfs/relocation.c | 24 ++-- 1 file changed, 22 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c index 63e416ac79ae..1f371a878831 100644 --- a/fs/btrfs/relocation.c +++ b/fs/btr

[PATCH v8 03/39] btrfs: handle errors from select_reloc_root()

2021-03-12 Thread Josef Bacik
indicate corruption. Signed-off-by: Josef Bacik --- fs/btrfs/relocation.c | 15 --- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c index 1f371a878831..097fea09e2d2 100644 --- a/fs/btrfs/relocation.c +++ b/fs/btrfs/rel

[PATCH v8 04/39] btrfs: convert BUG_ON()'s in select_reloc_root() to proper errors

2021-03-12 Thread Josef Bacik
generally indicates corruption, I've hit a few machines in the fleet that trip over these with clearly corrupted extent trees, so be nice and spit out an error message and return an error instead of bringing the whole box down. Signed-off-by: Josef Bacik --- fs/btrfs/re

[PATCH v8 01/39] btrfs: convert some BUG_ON()'s to ASSERT()'s in do_relocation

2021-03-12 Thread Josef Bacik
A few of these are checking for correctness, and won't be triggered by corrupted file systems, so convert them to ASSERT() instead of BUG_ON() and add a comment explaining their existence. Reviewed-by: Qu Wenruo Signed-off-by: Josef Bacik --- fs/btrfs/relocation.c | 19 -

[PATCH v8 00/39] Cleanup error handling in relocation

2021-03-12 Thread Josef Bacik
the series. I tested this with my error injection stress test, where I keep track of all stack traces that have been tested and only inject errors when we have a new stack trace, which means I should have covered all of the various error conditions. With this patchset I'm no longer panicing

Re: [PATCH v7 03/38] btrfs: handle errors from select_reloc_root()

2021-03-11 Thread Josef Bacik
On 2/26/21 1:30 PM, David Sterba wrote: On Wed, Dec 16, 2020 at 11:26:19AM -0500, Josef Bacik wrote: Currently select_reloc_root() doesn't return an error, but followup patches will make it possible for it to return an error. We do have proper error recovery in do_relocation however, so h

[PATCH 1/3] btrfs: init devices always

2021-03-11 Thread Josef Bacik
nally always so that we can properly deref device->fs_info in rescue cases. Reported-by: Neal Gompa Signed-off-by: Josef Bacik --- fs/btrfs/disk-io.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 41b718cfea40..63656bf23ff2 1

[PATCH 2/3] btrfs: do not init dev stats if we have no dev_root

2021-03-11 Thread Josef Bacik
g the device stats if we do not have a device root. Reported-by: Neal Gompa Signed-off-by: Josef Bacik --- fs/btrfs/volumes.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index 995920fcce9b..d4ca721c1d91 100644 --- a/fs/btrfs/volumes.c +++ b/fs/

[PATCH 3/3] btrfs: don't init dev replace for bad dev root

2021-03-11 Thread Josef Bacik
entry_SYSCALL_64_after_hwframe+0x44/0xa9 RIP: 0033:0x7f644730352e Fix this by not starting the device replace stuff if we do not have a NULL dev root. Reported-by: Neal Gompa Signed-off-by: Josef Bacik --- fs/btrfs/dev-replace.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/fs/btrfs/dev

[PATCH 0/3] Handle bad dev_root properly with rescue=all

2021-03-11 Thread Josef Bacik
ollowing 3 fixes allowed Neal to get rescue=all working without panicing the machine, and I verified everything by using btrfs-corrupt-block to corrupt a dev root of a file system. Thanks, Josef Josef Bacik (3): btrfs: init devices always btrfs: do not init dev stats if we have no dev_root bt

Re: Recovering Btrfs from a freak failure of the disk controller

2021-03-09 Thread Josef Bacik
On 3/9/21 4:06 PM, Neal Gompa wrote: On Tue, Mar 9, 2021 at 2:04 PM Josef Bacik wrote: On 3/8/21 8:12 PM, Neal Gompa wrote: On Mon, Mar 8, 2021 at 5:04 PM Josef Bacik wrote: On 3/8/21 3:01 PM, Neal Gompa wrote: On Mon, Mar 8, 2021 at 1:38 PM Josef Bacik wrote: On 3/5/21 8:03 PM, Neal

<    1   2   3   4   5   6   7   8   9   10   >