Re: Trying to work with the tests

2024-04-23 Thread Gao Xiang
Hi Ian, On 2024/4/22 21:10, Ian Kent wrote: On 22/4/24 17:12, Gao Xiang wrote: Hi Ian, (+Cc Jingbo here). On 2024/4/22 16:31, Ian Kent wrote: I'm new to the list so Hi to all, I'm working with a heavily patched 5.14 kernel and I've gathered together patches to bring erofs up to 5.19 and

Re: [PATCH -next v3 1/2] erofs: get rid of erofs_fs_context

2024-04-23 Thread Gao Xiang
On 2024/4/19 20:36, Baokun Li wrote: Instead of allocating the erofs_sb_info in fill_super() allocate it during erofs_init_fs_context() and ensure that erofs can always have the info available during erofs_kill_sb(). After this erofs_fs_context is no longer needed, replace ctx with sbi, no fun

Re: [syzbot] [erofs?] BUG: using smp_processor_id() in preemptible code in z_erofs_get_gbuf

2024-04-23 Thread Gao Xiang
On 2024/4/10 13:19, syzbot wrote: Hello, syzbot tried to test the proposed patch but the build/boot failed: #syz invalid Since https://lore.kernel.org/r/20240408215231.3376659-1-dhav...@google.com has been folded into the original patch and this issue is only in -next. Thanks, Gao Xiang

[xiang-erofs:dev-test] BUILD SUCCESS 569a48fed3558058620fed06a910f39e4ad82915

2024-04-23 Thread kernel test robot
-randconfig-001-20240423 clang i386 buildonly-randconfig-002-20240423 clang i386 buildonly-randconfig-003-20240423 gcc i386 buildonly-randconfig-004-20240423 clang i386 buildonly-randconfig-005-20240423 clang i386 buildonly-randconfig-006-20240423

Re: [PATCH -next] erofs: modify the error message when prepare_ondemand_read failed

2024-04-23 Thread Gao Xiang
(+cc linux-erofs & LKML) On 2024/4/24 10:39, Hongbo Li wrote: When prepare_ondemand_read failed, wrong error message is printed. The prepare_read is also implemented in cachefiles, so we amend it. Signed-off-by: Hongbo Li Reviewed-by: Gao Xiang Could you resend the patch with proper mail

[PATCH 2/5] cachefiles: flush all requests for the object that is being dropped

2024-04-23 Thread libaokun
From: Baokun Li Because after an object is dropped, requests for that object are useless, flush them to avoid causing other problems. Signed-off-by: Baokun Li --- fs/cachefiles/ondemand.c | 19 +++ 1 file changed, 19 insertions(+) diff --git a/fs/cachefiles/ondemand.c b/fs/cac

[PATCH 1/5] cachefiles: stop sending new request when dropping object

2024-04-23 Thread libaokun
From: Baokun Li Added CACHEFILES_ONDEMAND_OBJSTATE_DROPPING indicates that the cachefiles object is being dropped, and is set after the close request for the dropped object completes, and no new requests are allowed to be sent after this state. Signed-off-by: Baokun Li --- fs/cachefiles/intern

[PATCH 0/5] cachefiles: some bugfixes for clean object/send req/poll

2024-04-23 Thread libaokun
From: Baokun Li Hello everyone! Recently we found some bugs while doing tests on cachefiles ondemand mode, and this patchset is a fix for some of those issues. The following is a brief overview of the patches, see the patches for more details. Patch 1-3: After an object has been cleaned up, mak

[PATCH 3/5] cachefiles: flush ondemand_object_worker during clean object

2024-04-23 Thread libaokun
From: Hou Tao When queuing ondemand_object_worker() to re-open the object, cachefiles_object is not pinned. The cachefiles_object may be freed when the pending read request is completed intentionally and the related erofs is umounted. If ondemand_object_worker() runs after the object is freed, it

[PATCH 5/5] cachefiles: add missing lock protection when polling

2024-04-23 Thread libaokun
From: Jingbo Xu Add missing lock protection in poll routine when iterating xarray, otherwise: Even with RCU read lock held, only the slot of the radix tree is ensured to be pinned there, while the data structure (e.g. struct cachefiles_req) stored in the slot has no such guarantee. The poll rou

[PATCH 4/5] cachefiles: cyclic allocation of msg_id to avoid reuse

2024-04-23 Thread libaokun
From: Baokun Li Reusing the msg_id after a maliciously completed reopen request may cause a read request to remain unprocessed and result in a hung, as shown below: t1 | t2 | t3 - cachefiles_ondemand_select_req cachefi

[PATCH 02/12] cachefiles: remove err_put_fd tag in cachefiles_ondemand_daemon_read()

2024-04-23 Thread libaokun
From: Baokun Li The err_put_fd tag is only used once, so remove it to make the code more readable. Signed-off-by: Baokun Li --- fs/cachefiles/ondemand.c | 7 +++ 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/fs/cachefiles/ondemand.c b/fs/cachefiles/ondemand.c index 4ba42f1f

[PATCH 04/12] cachefiles: fix slab-use-after-free in cachefiles_ondemand_daemon_read()

2024-04-23 Thread libaokun
From: Baokun Li We got the following issue in a fuzz test of randomly issuing the restore command: == BUG: KASAN: slab-use-after-free in cachefiles_ondemand_daemon_read+0xb41/0xb60 Read of size 8 at addr 888122e84088 by task ond

[PATCH 05/12] cachefiles: add output string to cachefiles_obj_[get|put]_ondemand_fd

2024-04-23 Thread libaokun
From: Baokun Li This lets us see the correct trace output. Fixes: c8383054506c ("cachefiles: notify the user daemon when looking up cookie") Signed-off-by: Baokun Li --- include/trace/events/cachefiles.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/include/trace/events/cachefiles.h

[PATCH 01/12] cachefiles: remove request from xarry during flush requests

2024-04-23 Thread libaokun
From: Baokun Li This prevents concurrency from causing access to a freed req. Signed-off-by: Baokun Li --- fs/cachefiles/daemon.c | 1 + 1 file changed, 1 insertion(+) diff --git a/fs/cachefiles/daemon.c b/fs/cachefiles/daemon.c index 6465e2574230..ccb7b707ea4b 100644 --- a/fs/cachefiles/daem

[PATCH 03/12] cachefiles: fix slab-use-after-free in cachefiles_ondemand_get_fd()

2024-04-23 Thread libaokun
From: Baokun Li We got the following issue in a fuzz test of randomly issuing the restore command: == BUG: KASAN: slab-use-after-free in cachefiles_ondemand_daemon_read+0x609/0xab0 Write of size 4 at addr 888109164a80 by task on

[PATCH 06/12] cachefiles: add consistency check for copen/cread

2024-04-23 Thread libaokun
From: Baokun Li This prevents malicious processes from completing random copen/cread requests and crashing the system. Added checks are listed below: * Generic, copen can only complete open requests, and cread can only complete read requests. * For copen, ondemand_id must not be 0, becau

[PATCH 00/12] cachefiles: some bugfixes and cleanups for ondemand requests

2024-04-23 Thread libaokun
From: Baokun Li Hello everyone! Recently we found some bugs while doing tests on cachefiles ondemand mode, and this patchset is a fix for some of those issues. The following is a brief overview of the patches, see the patches for more details. Patch 1-5: Holding reference counts of reqs and obj

[PATCH 07/12] cachefiles: add spin_lock for cachefiles_ondemand_info

2024-04-23 Thread libaokun
From: Baokun Li The following concurrency may cause a read request to fail to be completed and result in a hung: t1 | t2 - cachefiles_ondemand_copen

[PATCH 09/12] cachefiles: defer exposing anon_fd until after copy_to_user() succeeds

2024-04-23 Thread libaokun
From: Baokun Li After installing the anonymous fd, we can now see it in userland and close it. However, at this point we may not have gotten the reference count of the cache, but we will put it during colse fd, so this may cause a cache UAF. To avoid this, we will make the anonymous fd accessibl

[PATCH 10/12] cachefiles: Set object to close if ondemand_id < 0 in copen

2024-04-23 Thread libaokun
From: Zizhi Wo If copen is maliciously called in the user mode, it may delete the request corresponding to the random id. And the request may have not been read yet. Note that when the object is set to reopen, the open request will be done with the still reopen state in above case. As a result,

[PATCH 08/12] cachefiles: never get a new anon fd if ondemand_id is valid

2024-04-23 Thread libaokun
From: Baokun Li Now every time the daemon reads an open request, it requests a new anon fd and ondemand_id. With the introduction of "restore", it is possible to read the same open request more than once, and therefore have multiple anon fd's for the same object. To avoid this, allocate a new an

[PATCH 12/12] cachefiles: make on-demand read killable

2024-04-23 Thread libaokun
From: Baokun Li Replacing wait_for_completion() with wait_for_completion_killable() in cachefiles_ondemand_send_req() allows us to kill processes that might trigger a hunk_task if the daemon is abnormal. But now only CACHEFILES_OP_READ is killable, because OP_CLOSE and OP_OPEN is initiated from

[PATCH 11/12] cachefiles: flush all requests after setting CACHEFILES_DEAD

2024-04-23 Thread libaokun
From: Baokun Li In ondemand mode, when the daemon is processing an open request, if the kernel flags the cache as CACHEFILES_DEAD, the cachefiles_daemon_write() will always return -EIO, so the daemon can't pass the copen to the kernel. Then the kernel process that is waiting for the copen trigger

[PATCH 4/5] cachefiles: correct the return value of cachefiles_check_volume_xattr()

2024-04-23 Thread libaokun
From: Baokun Li In cachefiles_check_volume_xattr(), the error returned by vfs_getxattr is not passed to ret, so it ends up returning -ESTALE, which leads to an endless loop as follows: cachefiles_acquire_volume retry: cachefiles_check_volume_xattr // return -ESTALE cachefiles_bury_object

[PATCH 2/5] cachefiles: fix slab-use-after-free in fscache_withdraw_volume()

2024-04-23 Thread libaokun
From: Baokun Li We got the following issue in our fault injection stress test: == BUG: KASAN: slab-use-after-free in fscache_withdraw_volume+0x2e1/0x370 Read of size 4 at addr 88810680be08 by task ondemand-04-dae/5798 CPU: 0 PI

[PATCH 3/5] cachefiles: fix slab-use-after-free in cachefiles_withdraw_cookie()

2024-04-23 Thread libaokun
From: Baokun Li We got the following issue in our fault injection stress test: == BUG: KASAN: slab-use-after-free in cachefiles_withdraw_cookie+0x4d9/0x600 Read of size 8 at addr 888118efc000 by task kworker/u78:0/109 CPU: 13 P

[PATCH 0/5] cachefiles: some bugfixes for withdraw and xattr

2024-04-23 Thread libaokun
From: Baokun Li Hello everyone! Recently we found some bugs while doing tests on cachefiles ondemand mode, and this patchset is a fix for some of those issues. The following is a brief overview of the patches, see the patches for more details. Patch 1-2: Add fscache_try_get_volume() helper func

[PATCH 5/5] cachefiles: correct the return value of cachefiles_check_auxdata()

2024-04-23 Thread libaokun
From: Baokun Li Pass the error code to ret when xlen < 0 to avoid misleading the caller. Fixes: 72b957856b0c ("cachefiles: Implement metadata/coherency data storage in xattrs") Signed-off-by: Baokun Li --- fs/cachefiles/xattr.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --

[PATCH 1/5] netfs, fscache: export fscache_put_volume() and add fscache_try_get_volume()

2024-04-23 Thread libaokun
From: Baokun Li Export fscache_put_volume() and add fscache_try_get_volume() helper function to allow cachefiles to get/put fscache_volume via linux/fscache-cache.h. Signed-off-by: Baokun Li --- fs/netfs/fscache_volume.c | 14 ++ fs/netfs/internal.h | 2 -- include/l

Re: [PATCH 5/5] cachefiles: add missing lock protection when polling

2024-04-23 Thread Gao Xiang
Hi Baokun, On 2024/4/24 11:34, libao...@huaweicloud.com wrote: From: Jingbo Xu Add missing lock protection in poll routine when iterating xarray, otherwise: Even with RCU read lock held, only the slot of the radix tree is ensured to be pinned there, while the data structure (e.g. struct cache

[PATCH] erofs-utils: add missing block counting

2024-04-23 Thread Noboru Asai
Add missing block counting when the data to be inlined is not inlined. Signed-off-by: Noboru Asai --- lib/inode.c | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/inode.c b/lib/inode.c index cf22bbe..727dcee 100644 --- a/lib/inode.c +++ b/lib/inode.c @@ -840,6 +840,7 @@ static int erofs_w

Re: [PATCH] erofs-utils: add missing block counting

2024-04-23 Thread Gao Xiang
Hi Noboru, On 2024/4/24 12:34, Noboru Asai wrote: Add missing block counting when the data to be inlined is not inlined. Signed-off-by: Noboru Asai Thanks for catching this! Could we fixup this at erofs_prepare_tail_block()? since currently it the place to allocate a tail block for this.

Re: [PATCH] erofs-utils: add missing block counting

2024-04-23 Thread Noboru Asai
Hi Gao, I think that erofs_balloc() and erofs_bh_baloon() function in erofs_write_tail_end() also alloc a tail block, Is it not true? 2024年4月24日(水) 13:54 Gao Xiang : > > Hi Noboru, > > On 2024/4/24 12:34, Noboru Asai wrote: > > Add missing block counting when the data to be inlined is not inlined

Re: [PATCH] erofs-utils: add missing block counting

2024-04-23 Thread Gao Xiang
On 2024/4/24 13:33, Noboru Asai wrote: Hi Gao, I think that erofs_balloc() and erofs_bh_baloon() function in erofs_write_tail_end() also alloc a tail block, Is it not true? erofs_prepare_tail_block() is the place to decide the fallback tail block. But due to some dependency, bh can be alloc

Re: [PATCH 5/5] cachefiles: add missing lock protection when polling

2024-04-23 Thread Jia Zhu via Linux-erofs
在 2024/4/24 11:34, libao...@huaweicloud.com 写道: From: Jingbo Xu Add missing lock protection in poll routine when iterating xarray, otherwise: Even with RCU read lock held, only the slot of the radix tree is ensured to be pinned there, while the data structure (e.g. struct cachefiles_req) st

[PATCH v2] erofs-utils: add missing block counting

2024-04-23 Thread Noboru Asai
Add missing block counting when the data to be inlined is not inlined. --- v2: - move from erofs_write_tail_end() to erofs_prepare_tail_block() Signed-off-by: Noboru Asai --- lib/inode.c | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/inode.c b/lib/inode.c index cf22bbe..9aba69d 100644

Re: [PATCH v2] erofs-utils: fsck: extract chunk-based file with hole correctly

2024-04-23 Thread Gao Xiang
On Mon, Apr 22, 2024 at 07:31:32PM +0800, Yifan Zhao wrote: > Currently fsck skips file extraction if it finds that EROFS_MAP_MAPPED > is unset, which is not the case for chunk-based files with hole. This > patch handles the corner case correctly. > > Signed-off-by: Yifan Zhao I will apply the f

Re: [PATCH v2] erofs-utils: add missing block counting

2024-04-23 Thread Gao Xiang
On 2024/4/24 13:59, Noboru Asai wrote: Add missing block counting when the data to be inlined is not inlined. --- v2: - move from erofs_write_tail_end() to erofs_prepare_tail_block() Signed-off-by: Noboru Asai Reviewed-by: Gao Xiang Thanks, Gao Xiang

Re: [PATCH 5/5] cachefiles: add missing lock protection when polling

2024-04-23 Thread Baokun Li
Hi Xiang, On 2024/4/24 12:29, Gao Xiang wrote: Hi Baokun, On 2024/4/24 11:34, libao...@huaweicloud.com wrote: From: Jingbo Xu Add missing lock protection in poll routine when iterating xarray, otherwise: Even with RCU read lock held, only the slot of the radix tree is ensured to be pinned t