[PATCH v4 14/33] afs: Fix EEXIST error returned from afs_rmdir() to be ENOTEMPTY

2024-11-08 Thread David Howells
() to translate EEXIST to ENOTEMPTY. Fixes: 260a980317da ("[AFS]: Add "directory write" support.") Signed-off-by: David Howells cc: Marc Dionne cc: linux-...@lists.infradead.org --- fs/afs/dir.c | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/fs/afs/d

[PATCH v4 21/33] netfs: Add support for caching single monolithic objects such as AFS dirs

2024-11-08 Thread David Howells
Add support for caching the content of a file that contains a single monolithic object that must be read/written with a single I/O operation, such as an AFS directory. Signed-off-by: David Howells cc: Jeff Layton cc: Marc Dionne cc: ne...@lists.linux.dev cc: linux-...@lists.infradead.org cc

[PATCH v4 01/33] kheaders: Ignore silly-rename files

2024-11-08 Thread David Howells
./include/linux/greybus/.__afs3C95: File removed before we read it The find warning doesn't seem to cause a problem. Fix this by telling tar when called from in gen_kheaders.sh to exclude such files. This only affects afs and nfs; cifs uses the Windows Hidden attribute to prevent the file from

[PATCH v4 33/33] netfs: Report on NULL folioq in netfs_writeback_unlock_folios()

2024-11-08 Thread David Howells
eturn. Note that this does not try to fix the problem. Reported-by: syzbot+af5c06208fa71bf31...@syzkaller.appspotmail.com Link: https://syzkaller.appspot.com/bug?extid=af5c06208fa71bf31b16 Signed-off-by: David Howells cc: Chang Yu Link: https://lore.kernel.org/r/zxshmew4u7mtg...@gmail.com/ cc:

[PATCH v4 32/33] afs: Add a tracepoint for afs_read_receive()

2024-11-08 Thread David Howells
Add a tracepoint for afs_read_receive() to allow potential missed wakeups to be debugged. Signed-off-by: David Howells cc: Marc Dionne cc: linux-...@lists.infradead.org --- fs/afs/file.c | 1 + include/trace/events/afs.h | 30 ++ 2 files changed, 31

[PATCH v4 31/33] afs: Locally initialise the contents of a new symlink on creation

2024-11-08 Thread David Howells
Since we know what the contents of a symlink will be when we create it on the server, initialise its contents locally too to avoid the need to download it. Signed-off-by: David Howells cc: Marc Dionne cc: linux-...@lists.infradead.org --- fs/afs/dir.c | 2 ++ fs/afs/inode.c

[PATCH v4 28/33] netfs: Change the read result collector to only use one work item

2024-11-08 Thread David Howells
ryption sequentially (though it's great if the decryption can happen in parallel). There is a disadvantage in that we're losing the ability to decrypt and unlock things on an as-things-arrive basis which may affect some applications. Signed-off-by: David Howells cc: Jeff Layton

[PATCH v4 30/33] afs: Use the contained hashtable to search a directory

2024-11-08 Thread David Howells
taken that the hash chains are acyclic. Signed-off-by: David Howells cc: Marc Dionne cc: linux-...@lists.infradead.org --- fs/afs/Makefile | 1 + fs/afs/dir.c| 42 fs/afs/dir_edit.c | 135 +- fs/afs/dir_search.c | 227

[PATCH v4 29/33] afs: Make afs_mkdir() locally initialise a new directory's content

2024-11-08 Thread David Howells
Initialise a new directory's content when it is created by mkdir locally rather than downloading the content from the server as we can predict what it's going to look like. Signed-off-by: David Howells cc: Marc Dionne cc: linux-...@lists.infradead.org --- fs/afs/dir.c

[PATCH v4 27/33] afs: Make {Y,}FS.FetchData an asynchronous operation

2024-11-08 Thread David Howells
f possible. The ->done() call op is also split with ->immediate_cancel() handling the cancellation on failure to begin the call and ->done() handling the rest. This means that the AFS async FetchData code doesn't try to terminate the netfs subrequest twice. Signed-off-by: David Howe

[PATCH v4 23/33] afs: Use netfslib for directories

2024-11-08 Thread David Howells
ect that isn't uploaded to the server. (10) Enable caching on directories. (11) Only set the upload key for writeback for regular files. Notes: (*) We keep the ->release_folio(), ->invalidate_folio() and ->migrate_folio() ops as we set the mapping pointer on the folio.

[PATCH v4 26/33] afs: Fix cleanup of immediately failed async calls

2024-11-08 Thread David Howells
"error_do_abort:" case of afs_make_call() abort the call and then abandon it to the notification handler. Fixes: 34fa47612bfe ("afs: Fix race in async call refcounting") Reported-by: Marc Dionne Signed-off-by: David Howells cc: linux-...@lists.infradead.org --- fs/afs/inte

[PATCH v4 25/33] afs: Eliminate afs_read

2024-11-08 Thread David Howells
need there to the afs_call struct (we may be given a different amount of data than what we asked for and have to track what remains of that) and using the netfs_io_subrequest directly instead. Signed-off-by: David Howells cc: Marc Dionne cc: linux-...@lists.infradead.org --- fs/afs/file.c

[PATCH v4 24/33] afs: Use netfslib for symlinks, allowing them to be cached

2024-11-08 Thread David Howells
Use netfslib to read symlinks, thereby allowing them to be cached by fscache and cachefiles. Signed-off-by: David Howells cc: Marc Dionne cc: Jeff Layton cc: linux-...@lists.infradead.org cc: ne...@lists.linux.dev cc: linux-fsde...@vger.kernel.org --- fs/afs/file.c | 32

[PATCH v4 22/33] afs: Make afs_init_request() get a key if not given a file

2024-11-08 Thread David Howells
the process's keyrings instead when setting up a read. Signed-off-by: David Howells cc: Marc Dionne cc: linux-...@lists.infradead.org --- fs/afs/file.c | 16 1 file changed, 16 insertions(+) diff --git a/fs/afs/file.c b/fs/afs/file.c index f717168da4ab..a9d98d18407c 100644 -

[PATCH v4 19/33] afs: Add more tracepoints to do with tracking validity

2024-11-08 Thread David Howells
n the DV is recorded on a vnode. (5) afs_dv_mismatch: Log when the DV recorded on a vnode plus the expected delta for the operation does not match the DV we got back from the server. Signed-off-by: David Howells cc: Marc Dionne cc: linux-...@lists.infradead.org --- fs/afs/callb

[PATCH v4 18/33] cachefiles: Add auxiliary data trace

2024-11-08 Thread David Howells
Add a display of the first 8 bytes of the downloaded auxiliary data and of the on-disk stored auxiliary data as these are used in coherency management. In the case of afs, this holds the data version number. Signed-off-by: David Howells cc: Jeff Layton cc: ne...@lists.linux.dev cc: linux-fsde

[PATCH v4 17/33] cachefiles: Add some subrequest tracepoints

2024-11-08 Thread David Howells
Add some tracepoints into the cachefiles write paths. Signed-off-by: David Howells cc: ne...@lists.linux.dev --- fs/cachefiles/io.c | 4 include/trace/events/netfs.h | 3 +++ 2 files changed, 7 insertions(+) diff --git a/fs/cachefiles/io.c b/fs/cachefiles/io.c index 6a821a959b59

[PATCH v4 20/33] netfs: Add functions to build/clean a buffer in a folio_queue

2024-11-08 Thread David Howells
will have trace lines emitted and will be accounted in the stats. Signed-off-by: David Howells cc: Jeff Layton cc: Marc Dionne cc: ne...@lists.linux.dev cc: linux-...@lists.infradead.org cc: linux-fsde...@vger.kernel.org --- fs/netfs/misc.c | 95

[PATCH v4 16/33] netfs: Remove some extraneous directory invalidations

2024-11-08 Thread David Howells
In the directory editing code, we shouldn't re-invalidate the directory if it is already invalidated. Signed-off-by: David Howells cc: Marc Dionne cc: linux-...@lists.infradead.org --- fs/afs/dir_edit.c | 22 +- 1 file changed, 9 insertions(+), 13 deletions(-) diff --

[PATCH v4 15/33] afs: Fix directory format encoding struct

2024-11-08 Thread David Howells
number of alloc_ctrs in the code anyway. In future, however, I should start using the hash table to try and speed up afs_lookup(). Fix this by using the correct constant to declare the counter array. Fixes: 4ea219a839bf ("afs: Split the directory content defs into a header") Signed-off-by: Dav

[PATCH v4 10/33] netfs: Drop the error arg from netfs_read_subreq_terminated()

2024-11-08 Thread David Howells
Drop the error argument from netfs_read_subreq_terminated() in favour of passing the value in subreq->error. Signed-off-by: David Howells cc: Jeff Layton cc: ne...@lists.linux.dev cc: linux-fsde...@vger.kernel.org --- fs/9p/vfs_addr.c | 3 ++- fs/afs/file.c|

[PATCH v4 13/33] afs: Don't use mutex for I/O operation lock

2024-11-08 Thread David Howells
data and not from the issuing thread. Signed-off-by: David Howells cc: Marc Dionne cc: linux-...@lists.infradead.org --- fs/afs/fs_operation.c | 111 +++--- fs/afs/internal.h | 3 +- fs/afs/super.c| 2 +- 3 files changed, 108 insertions(

[PATCH v4 12/33] netfs: Don't use bh spinlock

2024-11-08 Thread David Howells
All the accessing of the subrequest lists is now done in process context, possibly in a workqueue, but not now in a BH context, so we don't need the lock against BH interference when taking the netfs_io_request::lock spinlock. Signed-off-by: David Howells cc: Jeff Layton cc: linux

[PATCH v4 11/33] netfs: Drop the was_async arg from netfs_read_subreq_terminated()

2024-11-08 Thread David Howells
-=wiVC5Cgyz6QKXFu6fTaA6h4CjexDR-OV9kL6Vo5x9v8=a...@mail.gmail.com/ Signed-off-by: David Howells cc: Jeff Layton cc: ne...@lists.linux.dev cc: linux-fsde...@vger.kernel.org --- fs/9p/vfs_addr.c | 2 +- fs/afs/file.c| 6 ++--- fs/afs/fsclient.c| 2 +- fs/afs/yfsclient.c

[PATCH v4 09/33] netfs: Split retry code out of fs/netfs/write_collect.c

2024-11-08 Thread David Howells
Split write-retry code out of fs/netfs/write_collect.c as it will become more elaborate when content crypto is introduced. Signed-off-by: David Howells cc: Jeff Layton cc: ne...@lists.linux.dev cc: linux-fsde...@vger.kernel.org --- fs/netfs/Makefile| 3 +- fs/netfs/internal.h

[PATCH v4 08/33] netfs: Make netfs_advance_write() return size_t

2024-11-08 Thread David Howells
netfs_advance_write() calculates the amount of data it's attaching to a stream with size_t, but then returns this as an int. Switch the return value to size_t for consistency. Signed-off-by: David Howells cc: Jeff Layton cc: linux-cach...@redhat.com cc: linux-fsde...@vger.kernel.org cc:

[PATCH v4 07/33] netfs: Abstract out a rolling folio buffer implementation

2024-11-08 Thread David Howells
use a second rolling buffer to handle bounce buffering for content encryption. Signed-off-by: David Howells cc: Jeff Layton cc: ne...@lists.linux.dev cc: linux-fsde...@vger.kernel.org --- fs/netfs/Makefile | 1 + fs/netfs/buffered_read.c | 119 - fs/

[PATCH v4 06/33] netfs: Add a tracepoint to log the lifespan of folio_queue structs

2024-11-08 Thread David Howells
Add a tracepoint to log the lifespan of folio_queue structs. For tracing illustrative purposes, folio_queues are tagged with the debug ID of whatever they're related to (typically a netfs_io_request) and a debug ID of their own. Signed-off-by: David Howells cc: Jeff Layton c

[PATCH v4 02/33] netfs: Remove call to folio_index()

2024-11-08 Thread David Howells
From: "Matthew Wilcox (Oracle)" Calling folio_index() is pointless overhead; directly dereferencing folio->index is fine. Signed-off-by: Matthew Wilcox (Oracle) Link: https://lore.kernel.org/r/20241005182307.3190401-2-wi...@infradead.org Signed-off-by: Christian Brauner --- include/trace/even

[PATCH v4 03/33] netfs: Fix a few minor bugs in netfs_page_mkwrite()

2024-11-08 Thread David Howells
From: "Matthew Wilcox (Oracle)" We can't return with VM_FAULT_SIGBUS | VM_FAULT_LOCKED; the core code will not unlock the folio in this instance. Introduce a new "unlock" error exit to handle this case. Use it to handle the "folio is truncated" check, and change the "writeback interrupted by a

[PATCH v4 05/33] netfs: Use a folio_queue allocation and free functions

2024-11-08 Thread David Howells
Provide and use folio_queue allocation and free functions to combine the allocation, initialisation and stat (un)accounting steps that are repeated in several places. Signed-off-by: David Howells cc: Jeff Layton cc: ne...@lists.linux.dev cc: linux-fsde...@vger.kernel.org --- fs/netfs

[PATCH v4 04/33] netfs: Remove unnecessary references to pages

2024-11-08 Thread David Howells
From: "Matthew Wilcox (Oracle)" These places should all use folios instead of pages. Signed-off-by: Matthew Wilcox (Oracle) Link: https://lore.kernel.org/r/20241005182307.3190401-4-wi...@infradead.org Signed-off-by: Christian Brauner --- fs/netfs/buffered_read.c | 8 fs/netfs/buffe

[PATCH v4 00/33] netfs: Read performance improvements and "single-blob" support

2024-11-08 Thread David Howells
ectory's hash table on lookup. - Add an afs patch to preset the contents of a new symlink on creation. - Add an afs patch to add a tracepoint in the async FetchData response processing. - Add a patch to report if a NULL folio_queue pointer is seen in netfs_writeback_unlock_folios(). T

Re: [PATCH v3 23/33] afs: Use netfslib for directories

2024-11-08 Thread David Howells
The attached fix needs folding in across this patch (23), patch 24, patch 29 and patch 31. David --- commit 9d4429bc7bb3f2b518d6decd1ca0e99e4d80d58e Author: David Howells Date: Thu Nov 7 23:46:48 2024 + afs: Fix handling of signals during readdir When a directory is being

Re: [PATCH v3 28/33] netfs: Change the read result collector to only use one work item

2024-11-08 Thread David Howells
This patch needs the attached adjustment folding in. David --- commit 2c07b29a051fadb6816d31f526e4dd45ddf5 Author: David Howells Date: Thu Nov 7 22:22:49 2024 + netfs: Fix folio abandonment The mechanism to handle the abandonment of a folio being read due to an error

[PATCH v3 04/33] netfs: Remove unnecessary references to pages

2024-11-06 Thread David Howells
From: "Matthew Wilcox (Oracle)" These places should all use folios instead of pages. Signed-off-by: Matthew Wilcox (Oracle) Link: https://lore.kernel.org/r/20241005182307.3190401-4-wi...@infradead.org Signed-off-by: Christian Brauner --- fs/netfs/buffered_read.c | 8 fs/netfs/buffe

[PATCH v3 20/33] netfs: Add functions to build/clean a buffer in a folio_queue

2024-11-06 Thread David Howells
will have trace lines emitted and will be accounted in the stats. Signed-off-by: David Howells cc: Jeff Layton cc: Marc Dionne cc: ne...@lists.linux.dev cc: linux-...@lists.infradead.org cc: linux-fsde...@vger.kernel.org --- fs/netfs/misc.c | 95

[PATCH v3 09/33] netfs: Split retry code out of fs/netfs/write_collect.c

2024-11-06 Thread David Howells
Split write-retry code out of fs/netfs/write_collect.c as it will become more elaborate when content crypto is introduced. Signed-off-by: David Howells cc: Jeff Layton cc: ne...@lists.linux.dev cc: linux-fsde...@vger.kernel.org --- fs/netfs/Makefile| 3 +- fs/netfs/internal.h

[PATCH v3 10/33] netfs: Drop the error arg from netfs_read_subreq_terminated()

2024-11-06 Thread David Howells
Drop the error argument from netfs_read_subreq_terminated() in favour of passing the value in subreq->error. Signed-off-by: David Howells cc: Jeff Layton cc: ne...@lists.linux.dev cc: linux-fsde...@vger.kernel.org --- fs/9p/vfs_addr.c | 3 ++- fs/afs/file.c|

[PATCH v3 12/33] netfs: Don't use bh spinlock

2024-11-06 Thread David Howells
All the accessing of the subrequest lists is now done in process context, possibly in a workqueue, but not now in a BH context, so we don't need the lock against BH interference when taking the netfs_io_request::lock spinlock. Signed-off-by: David Howells cc: Jeff Layton cc: linux

[PATCH v3 00/33] netfs: Read performance improvements and "single-blob" support

2024-11-06 Thread David Howells
to search the directory's hash table on lookup. - Add an afs patch to preset the contents of a new symlink on creation. - Add an afs patch to add a tracepoint in the async FetchData response processing. - Add a patch to report if a NULL folio_queue pointer is seen in netfs_writeback_

[PATCH v3 05/33] netfs: Use a folio_queue allocation and free functions

2024-11-06 Thread David Howells
Provide and use folio_queue allocation and free functions to combine the allocation, initialisation and stat (un)accounting steps that are repeated in several places. Signed-off-by: David Howells cc: Jeff Layton cc: ne...@lists.linux.dev cc: linux-fsde...@vger.kernel.org --- fs/netfs

[PATCH v3 29/33] afs: Make afs_mkdir() locally initialise a new directory's content

2024-11-06 Thread David Howells
Initialise a new directory's content when it is created by mkdir locally rather than downloading the content from the server as we can predict what it's going to look like. Signed-off-by: David Howells cc: Marc Dionne cc: linux-...@lists.infradead.org --- fs/afs/dir.c

[PATCH v3 02/33] netfs: Remove call to folio_index()

2024-11-06 Thread David Howells
From: "Matthew Wilcox (Oracle)" Calling folio_index() is pointless overhead; directly dereferencing folio->index is fine. Signed-off-by: Matthew Wilcox (Oracle) Link: https://lore.kernel.org/r/20241005182307.3190401-2-wi...@infradead.org Signed-off-by: Christian Brauner --- include/trace/even

[PATCH v3 01/33] kheaders: Ignore silly-rename files

2024-11-06 Thread David Howells
./include/linux/greybus/.__afs3C95: File removed before we read it The find warning doesn't seem to cause a problem. Fix this by telling tar when called from in gen_kheaders.sh to exclude such files. This only affects afs and nfs; cifs uses the Windows Hidden attribute to prevent the file from

[PATCH v3 03/33] netfs: Fix a few minor bugs in netfs_page_mkwrite()

2024-11-06 Thread David Howells
From: "Matthew Wilcox (Oracle)" We can't return with VM_FAULT_SIGBUS | VM_FAULT_LOCKED; the core code will not unlock the folio in this instance. Introduce a new "unlock" error exit to handle this case. Use it to handle the "folio is truncated" check, and change the "writeback interrupted by a

[PATCH v3 13/33] afs: Don't use mutex for I/O operation lock

2024-11-06 Thread David Howells
data and not from the issuing thread. Signed-off-by: David Howells cc: Marc Dionne cc: linux-...@lists.infradead.org --- fs/afs/fs_operation.c | 111 +++--- fs/afs/internal.h | 3 +- fs/afs/super.c| 2 +- 3 files changed, 108 insertions(

[PATCH v3 08/33] netfs: Make netfs_advance_write() return size_t

2024-11-06 Thread David Howells
netfs_advance_write() calculates the amount of data it's attaching to a stream with size_t, but then returns this as an int. Switch the return value to size_t for consistency. Signed-off-by: David Howells cc: Jeff Layton cc: linux-cach...@redhat.com cc: linux-fsde...@vger.kernel.org cc:

[PATCH v3 19/33] afs: Add more tracepoints to do with tracking validity

2024-11-06 Thread David Howells
n the DV is recorded on a vnode. (5) afs_dv_mismatch: Log when the DV recorded on a vnode plus the expected delta for the operation does not match the DV we got back from the server. Signed-off-by: David Howells cc: Marc Dionne cc: linux-...@lists.infradead.org --- fs/afs/callb

[PATCH v3 30/33] afs: Use the contained hashtable to search a directory

2024-11-06 Thread David Howells
taken that the hash chains are acyclic. Signed-off-by: David Howells cc: Marc Dionne cc: linux-...@lists.infradead.org --- fs/afs/Makefile | 1 + fs/afs/dir.c| 42 fs/afs/dir_edit.c | 135 +- fs/afs/dir_search.c | 227

[PATCH v3 32/33] afs: Add a tracepoint for afs_read_receive()

2024-11-06 Thread David Howells
Add a tracepoint for afs_read_receive() to allow potential missed wakeups to be debugged. Signed-off-by: David Howells cc: Marc Dionne cc: linux-...@lists.infradead.org --- fs/afs/file.c | 1 + include/trace/events/afs.h | 30 ++ 2 files changed, 31

[PATCH v3 27/33] afs: Make {Y,}FS.FetchData an asynchronous operation

2024-11-06 Thread David Howells
f possible. The ->done() call op is also split with ->immediate_cancel() handling the cancellation on failure to begin the call and ->done() handling the rest. This means that the AFS async FetchData code doesn't try to terminate the netfs subrequest twice. Signed-off-by: David Howe

[PATCH v3 33/33] netfs: Report on NULL folioq in netfs_writeback_unlock_folios()

2024-11-06 Thread David Howells
eturn. Note that this does not try to fix the problem. Reported-by: syzbot+af5c06208fa71bf31...@syzkaller.appspotmail.com Link: https://syzkaller.appspot.com/bug?extid=af5c06208fa71bf31b16 Signed-off-by: David Howells cc: Chang Yu Link: https://lore.kernel.org/r/zxshmew4u7mtg...@gmail.com/ cc:

[PATCH v3 31/33] afs: Locally initialise the contents of a new symlink on creation

2024-11-06 Thread David Howells
Since we know what the contents of a symlink will be when we create it on the server, initialise its contents locally too to avoid the need to download it. Signed-off-by: David Howells cc: Marc Dionne cc: linux-...@lists.infradead.org --- fs/afs/dir.c | 2 ++ fs/afs/inode.c

[PATCH v3 18/33] cachefiles: Add auxiliary data trace

2024-11-06 Thread David Howells
Add a display of the first 8 bytes of the downloaded auxiliary data and of the on-disk stored auxiliary data as these are used in coherency management. In the case of afs, this holds the data version number. Signed-off-by: David Howells cc: Jeff Layton cc: ne...@lists.linux.dev cc: linux-fsde

[PATCH v3 22/33] afs: Make afs_init_request() get a key if not given a file

2024-11-06 Thread David Howells
the process's keyrings instead when setting up a read. Signed-off-by: David Howells cc: Marc Dionne cc: linux-...@lists.infradead.org --- fs/afs/file.c | 16 1 file changed, 16 insertions(+) diff --git a/fs/afs/file.c b/fs/afs/file.c index f717168da4ab..a9d98d18407c 100644 -

[PATCH v3 28/33] netfs: Change the read result collector to only use one work item

2024-11-06 Thread David Howells
ryption sequentially (though it's great if the decryption can happen in parallel). There is a disadvantage in that we're losing the ability to decrypt and unlock things on an as-things-arrive basis which may affect some applications. Signed-off-by: David Howells cc: Jeff Layton

[PATCH v3 07/33] netfs: Abstract out a rolling folio buffer implementation

2024-11-06 Thread David Howells
use a second rolling buffer to handle bounce buffering for content encryption. Signed-off-by: David Howells cc: Jeff Layton cc: ne...@lists.linux.dev cc: linux-fsde...@vger.kernel.org --- fs/netfs/Makefile | 1 + fs/netfs/buffered_read.c | 119 - fs/

[PATCH v3 16/33] netfs: Remove some extraneous directory invalidations

2024-11-06 Thread David Howells
In the directory editing code, we shouldn't re-invalidate the directory if it is already invalidated. Signed-off-by: David Howells cc: Marc Dionne cc: linux-...@lists.infradead.org --- fs/afs/dir_edit.c | 22 +- 1 file changed, 9 insertions(+), 13 deletions(-) diff --

[PATCH v3 17/33] cachefiles: Add some subrequest tracepoints

2024-11-06 Thread David Howells
Add some tracepoints into the cachefiles write paths. Signed-off-by: David Howells cc: ne...@lists.linux.dev --- fs/cachefiles/io.c | 4 include/trace/events/netfs.h | 3 +++ 2 files changed, 7 insertions(+) diff --git a/fs/cachefiles/io.c b/fs/cachefiles/io.c index 6a821a959b59

[PATCH v3 26/33] afs: Fix cleanup of immediately failed async calls

2024-11-06 Thread David Howells
"error_do_abort:" case of afs_make_call() abort the call and then abandon it to the notification handler. Fixes: 34fa47612bfe ("afs: Fix race in async call refcounting") Reported-by: Marc Dionne Signed-off-by: David Howells cc: linux-...@lists.infradead.org --- fs/afs/inte

[PATCH v3 21/33] netfs: Add support for caching single monolithic objects such as AFS dirs

2024-11-06 Thread David Howells
Add support for caching the content of a file that contains a single monolithic object that must be read/written with a single I/O operation, such as an AFS directory. Signed-off-by: David Howells cc: Jeff Layton cc: Marc Dionne cc: ne...@lists.linux.dev cc: linux-...@lists.infradead.org cc

[PATCH v3 24/33] afs: Use netfslib for symlinks, allowing them to be cached

2024-11-06 Thread David Howells
Use netfslib to read symlinks, thereby allowing them to be cached by fscache and cachefiles. Signed-off-by: David Howells cc: Marc Dionne cc: Jeff Layton cc: linux-...@lists.infradead.org cc: ne...@lists.linux.dev cc: linux-fsde...@vger.kernel.org --- fs/afs/file.c | 32

[PATCH v3 06/33] netfs: Add a tracepoint to log the lifespan of folio_queue structs

2024-11-06 Thread David Howells
Add a tracepoint to log the lifespan of folio_queue structs. For tracing illustrative purposes, folio_queues are tagged with the debug ID of whatever they're related to (typically a netfs_io_request) and a debug ID of their own. Signed-off-by: David Howells cc: Jeff Layton c

[PATCH v3 23/33] afs: Use netfslib for directories

2024-11-06 Thread David Howells
ect that isn't uploaded to the server. (10) Enable caching on directories. (11) Only set the upload key for writeback for regular files. Notes: (*) We keep the ->release_folio(), ->invalidate_folio() and ->migrate_folio() ops as we set the mapping pointer on the folio.

[PATCH v3 25/33] afs: Eliminate afs_read

2024-11-06 Thread David Howells
need there to the afs_call struct (we may be given a different amount of data than what we asked for and have to track what remains of that) and using the netfs_io_subrequest directly instead. Signed-off-by: David Howells cc: Marc Dionne cc: linux-...@lists.infradead.org --- fs/afs/file.c

[PATCH v3 14/33] afs: Fix EEXIST error returned from afs_rmdir() to be ENOTEMPTY

2024-11-06 Thread David Howells
() to translate EEXIST to ENOTEMPTY. Fixes: 260a980317da ("[AFS]: Add "directory write" support.") Signed-off-by: David Howells cc: Marc Dionne cc: linux-...@lists.infradead.org --- fs/afs/dir.c | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/fs/afs/d

[PATCH v3 15/33] afs: Fix directory format encoding struct

2024-11-06 Thread David Howells
number of alloc_ctrs in the code anyway. In future, however, I should start using the hash table to try and speed up afs_lookup(). Fix this by using the correct constant to declare the counter array. Fixes: 4ea219a839bf ("afs: Split the directory content defs into a header") Signed-off-by: Dav

[PATCH v3 11/33] netfs: Drop the was_async arg from netfs_read_subreq_terminated()

2024-11-06 Thread David Howells
-=wiVC5Cgyz6QKXFu6fTaA6h4CjexDR-OV9kL6Vo5x9v8=a...@mail.gmail.com/ Signed-off-by: David Howells cc: Jeff Layton cc: ne...@lists.linux.dev cc: linux-fsde...@vger.kernel.org --- fs/9p/vfs_addr.c | 2 +- fs/afs/file.c| 6 ++--- fs/afs/fsclient.c| 2 +- fs/afs/yfsclient.c

Re: [PATCH v2 25/31] afs: Make {Y,}FS.FetchData an asynchronous operation

2024-10-31 Thread David Howells
e server. (4) Implement this for FetchData RPCs, making it schedule the async handler and wait for it so that it doesn't get cancelled. Signed-off-by: David Howells cc: Marc Dionne cc: linux-...@lists.infradead.org --- fs/afs/file.c |8 fs/afs/fsclient.c |3 +++

[PATCH v2 00/31] netfs: Read performance improvements and "single-blob" support

2024-10-25 Thread David Howells
etchData response processing. - Add a patch to report if a NULL folio_queue pointer is seen in netfs_writeback_unlock_folios(). Thanks, David David Howells (28): netfs: Use a folio_queue allocation and free functions netfs: Add a tracepoint to log the lifespan of folio_queue structs ne

[PATCH v2 15/31] netfs: Remove some extraneous directory invalidations

2024-10-25 Thread David Howells
In the directory editing code, we shouldn't re-invalidate the directory if it is already invalidated. Signed-off-by: David Howells cc: Marc Dionne cc: linux-...@lists.infradead.org --- fs/afs/dir_edit.c | 22 +- 1 file changed, 9 insertions(+), 13 deletions(-) diff --

[PATCH v2 06/31] netfs: Abstract out a rolling folio buffer implementation

2024-10-25 Thread David Howells
use a second rolling buffer to handle bounce buffering for content encryption. Signed-off-by: David Howells cc: Jeff Layton cc: ne...@lists.linux.dev cc: linux-fsde...@vger.kernel.org --- fs/netfs/Makefile | 1 + fs/netfs/buffered_read.c | 119 - fs/

[PATCH v2 01/31] netfs: Remove call to folio_index()

2024-10-25 Thread David Howells
From: "Matthew Wilcox (Oracle)" Calling folio_index() is pointless overhead; directly dereferencing folio->index is fine. Signed-off-by: Matthew Wilcox (Oracle) Link: https://lore.kernel.org/r/20241005182307.3190401-2-wi...@infradead.org Signed-off-by: Christian Brauner --- include/trace/even

[PATCH v2 02/31] netfs: Fix a few minor bugs in netfs_page_mkwrite()

2024-10-25 Thread David Howells
From: "Matthew Wilcox (Oracle)" We can't return with VM_FAULT_SIGBUS | VM_FAULT_LOCKED; the core code will not unlock the folio in this instance. Introduce a new "unlock" error exit to handle this case. Use it to handle the "folio is truncated" check, and change the "writeback interrupted by a

[PATCH v2 03/31] netfs: Remove unnecessary references to pages

2024-10-25 Thread David Howells
From: "Matthew Wilcox (Oracle)" These places should all use folios instead of pages. Signed-off-by: Matthew Wilcox (Oracle) Link: https://lore.kernel.org/r/20241005182307.3190401-4-wi...@infradead.org Signed-off-by: Christian Brauner --- fs/netfs/buffered_read.c | 8 fs/netfs/buffe

[PATCH v2 31/31] netfs: Report on NULL folioq in netfs_writeback_unlock_folios()

2024-10-25 Thread David Howells
eturn. Note that this does not try to fix the problem. Reported-by: syzbot+af5c06208fa71bf31...@syzkaller.appspotmail.com Link: https://syzkaller.appspot.com/bug?extid=af5c06208fa71bf31b16 Signed-off-by: David Howells cc: Chang Yu Link: https://lore.kernel.org/r/zxshmew4u7mtg...@gmail.com/ cc:

[PATCH v2 30/31] afs: Add a tracepoint for afs_read_receive()

2024-10-25 Thread David Howells
Add a tracepoint for afs_read_receive() to allow potential missed wakeups to be debugged. Signed-off-by: David Howells cc: Marc Dionne cc: linux-...@lists.infradead.org --- fs/afs/file.c | 1 + include/trace/events/afs.h | 30 ++ 2 files changed, 31

[PATCH v2 28/31] afs: Use the contained hashtable to search a directory

2024-10-25 Thread David Howells
taken that the hash chains are acyclic. Signed-off-by: David Howells cc: Marc Dionne cc: linux-...@lists.infradead.org --- fs/afs/Makefile | 1 + fs/afs/dir.c| 42 fs/afs/dir_edit.c | 133 +- fs/afs/dir_search.c | 227

[PATCH v2 29/31] afs: Locally initialise the contents of a new symlink on creation

2024-10-25 Thread David Howells
Since we know what the contents of a symlink will be when we create it on the server, initialise its contents locally too to avoid the need to download it. Signed-off-by: David Howells cc: Marc Dionne cc: linux-...@lists.infradead.org --- fs/afs/dir.c | 2 ++ fs/afs/inode.c

[PATCH v2 27/31] afs: Make afs_mkdir() locally initialise a new directory's content

2024-10-25 Thread David Howells
Initialise a new directory's content when it is created by mkdir locally rather than downloading the content from the server as we can predict what it's going to look like. Signed-off-by: David Howells cc: Marc Dionne cc: linux-...@lists.infradead.org --- fs/afs/dir.c

[PATCH v2 26/31] netfs: Change the read result collector to only use one work item

2024-10-25 Thread David Howells
ryption sequentially (though it's great if the decryption can happen in parallel). There is a disadvantage in that we're losing the ability to decrypt and unlock things on an as-things-arrive basis which may affect some applications. Signed-off-by: David Howells cc: Jeff Layton

[PATCH v2 25/31] afs: Make {Y,}FS.FetchData an asynchronous operation

2024-10-25 Thread David Howells
f possible. Signed-off-by: David Howells cc: Marc Dionne cc: linux-...@lists.infradead.org --- fs/afs/file.c | 115 +- fs/afs/fs_operation.c | 2 +- fs/afs/fsclient.c | 6 ++- fs/afs/internal.h | 7 +++ fs/afs/main.c | 2

[PATCH v2 21/31] afs: Make afs_init_request() get a key if not given a file

2024-10-25 Thread David Howells
the process's keyrings instead when setting up a read. Signed-off-by: David Howells cc: Marc Dionne cc: linux-...@lists.infradead.org --- fs/afs/file.c | 16 1 file changed, 16 insertions(+) diff --git a/fs/afs/file.c b/fs/afs/file.c index f717168da4ab..a9d98d18407c 100644 -

[PATCH v2 24/31] afs: Eliminate afs_read

2024-10-25 Thread David Howells
need there to the afs_call struct (we may be given a different amount of data than what we asked for and have to track what remains of that) and using the netfs_io_subrequest directly instead. Signed-off-by: David Howells cc: Marc Dionne cc: linux-...@lists.infradead.org --- fs/afs/file.c

[PATCH v2 23/31] afs: Use netfslib for symlinks, allowing them to be cached

2024-10-25 Thread David Howells
Use netfslib to read symlinks, thereby allowing them to be cached by fscache and cachefiles. Signed-off-by: David Howells cc: Marc Dionne cc: Jeff Layton cc: linux-...@lists.infradead.org cc: ne...@lists.linux.dev cc: linux-fsde...@vger.kernel.org --- fs/afs/file.c | 32

[PATCH v2 22/31] afs: Use netfslib for directories

2024-10-25 Thread David Howells
ect that isn't uploaded to the server. (10) Enable caching on directories. (11) Only set the upload key for writeback for regular files. Notes: (*) We keep the ->release_folio(), ->invalidate_folio() and ->migrate_folio() ops as we set the mapping pointer on the folio.

[PATCH v2 20/31] netfs: Add support for caching single monolithic objects such as AFS dirs

2024-10-25 Thread David Howells
Add support for caching the content of a file that contains a single monolithic object that must be read/written with a single I/O operation, such as an AFS directory. Signed-off-by: David Howells cc: Jeff Layton cc: Marc Dionne cc: ne...@lists.linux.dev cc: linux-...@lists.infradead.org cc

[PATCH v2 19/31] netfs: Add functions to build/clean a buffer in a folio_queue

2024-10-25 Thread David Howells
will have trace lines emitted and will be accounted in the stats. Signed-off-by: David Howells cc: Jeff Layton cc: Marc Dionne cc: ne...@lists.linux.dev cc: linux-...@lists.infradead.org cc: linux-fsde...@vger.kernel.org --- fs/netfs/misc.c | 95

[PATCH v2 18/31] afs: Add more tracepoints to do with tracking validity

2024-10-25 Thread David Howells
n the DV is recorded on a vnode. (5) afs_dv_mismatch: Log when the DV recorded on a vnode plus the expected delta for the operation does not match the DV we got back from the server. Signed-off-by: David Howells cc: Marc Dionne cc: linux-...@lists.infradead.org --- fs/afs/callb

[PATCH v2 13/31] afs: Fix EEXIST error returned from afs_rmdir() to be ENOTEMPTY

2024-10-25 Thread David Howells
() to translate EEXIST to ENOTEMPTY. Fixes: 260a980317da ("[AFS]: Add "directory write" support.") Signed-off-by: David Howells cc: Marc Dionne cc: linux-...@lists.infradead.org --- fs/afs/dir.c | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/fs/afs/d

[PATCH v2 14/31] afs: Fix directory format encoding struct

2024-10-25 Thread David Howells
number of alloc_ctrs in the code anyway. In future, however, I should start using the hash table to try and speed up afs_lookup(). Fix this by using the correct constant to declare the counter array. Fixes: 4ea219a839bf ("afs: Split the directory content defs into a header") Signed-off-by: Dav

[PATCH v2 17/31] cachefiles: Add auxiliary data trace

2024-10-25 Thread David Howells
Add a display of the first 8 bytes of the downloaded auxiliary data and of the on-disk stored auxiliary data as these are used in coherency management. In the case of afs, this holds the data version number. Signed-off-by: David Howells cc: Jeff Layton cc: ne...@lists.linux.dev cc: linux-fsde

[PATCH v2 16/31] cachefiles: Add some subrequest tracepoints

2024-10-25 Thread David Howells
Add some tracepoints into the cachefiles write paths. Signed-off-by: David Howells cc: ne...@lists.linux.dev --- fs/cachefiles/io.c | 4 include/trace/events/netfs.h | 3 +++ 2 files changed, 7 insertions(+) diff --git a/fs/cachefiles/io.c b/fs/cachefiles/io.c index 6a821a959b59

[PATCH v2 12/31] afs: Don't use mutex for I/O operation lock

2024-10-25 Thread David Howells
data and not from the issuing thread. Signed-off-by: David Howells cc: Marc Dionne cc: linux-...@lists.infradead.org --- fs/afs/fs_operation.c | 111 +++--- fs/afs/internal.h | 3 +- fs/afs/super.c| 2 +- 3 files changed, 108 insertions(

[PATCH v2 11/31] netfs: Don't use bh spinlock

2024-10-25 Thread David Howells
All the accessing of the subrequest lists is now done in process context, possibly in a workqueue, but not now in a BH context, so we don't need the lock against BH interference when taking the netfs_io_request::lock spinlock. Signed-off-by: David Howells cc: Jeff Layton cc: linux

[PATCH v2 10/31] netfs: Drop the was_async arg from netfs_read_subreq_terminated()

2024-10-25 Thread David Howells
-=wiVC5Cgyz6QKXFu6fTaA6h4CjexDR-OV9kL6Vo5x9v8=a...@mail.gmail.com/ Signed-off-by: David Howells cc: Jeff Layton cc: ne...@lists.linux.dev cc: linux-fsde...@vger.kernel.org --- fs/9p/vfs_addr.c | 2 +- fs/afs/file.c| 6 ++--- fs/afs/fsclient.c| 2 +- fs/afs/yfsclient.c

[PATCH v2 09/31] netfs: Drop the error arg from netfs_read_subreq_terminated()

2024-10-25 Thread David Howells
Drop the error argument from netfs_read_subreq_terminated() in favour of passing the value in subreq->error. Signed-off-by: David Howells cc: Jeff Layton cc: ne...@lists.linux.dev cc: linux-fsde...@vger.kernel.org --- fs/9p/vfs_addr.c | 3 ++- fs/afs/file.c|

[PATCH v2 08/31] netfs: Split retry code out of fs/netfs/write_collect.c

2024-10-25 Thread David Howells
Split write-retry code out of fs/netfs/write_collect.c as it will become more elaborate when content crypto is introduced. Signed-off-by: David Howells cc: Jeff Layton cc: ne...@lists.linux.dev cc: linux-fsde...@vger.kernel.org --- fs/netfs/Makefile| 3 +- fs/netfs/internal.h

  1   2   3   4   5   >