How to display a ktime value as trace timestamp in trace output?

2024-01-31 Thread David Howells
Hi Steven, I have a tracepoint in AF_RXRPC that displays information about a timeout I'm going to set. I have the timeout in a ktime_t as an absolute time. Is there a way to display this in the trace output such that it looks like a trace timestamp and can be (roughly) correlated with the

Re: [PATCH 02/20] filelock: add coccinelle scripts to move fields to struct file_lock_core

2024-01-17 Thread David Howells
Do we need to keep these coccinelle scripts for posterity? Or can they just be included in the patch description of the patch that generates them? David

Re: [RFC][PATCH] fix short copy handling in copy_mc_pipe_to_iter()

2022-06-14 Thread David Howells
Al Viro wrote: > What's wrong with > p_occupancy = pipe_occupancy(head, tail); > if (p_occupancy >= pipe->max_usage) > return 0; > else > return pipe->max_usage - p_occupancy; Because "pipe->max_usage - p_occupancy" can be negative.

Fix for CVE-2020-26541

2021-04-16 Thread David Howells
Hi Linus, I posted a pull request for a fix for CVE-2020-26541: https://lore.kernel.org/keyrings/1884195.1615482...@warthog.procyon.org.uk/ [GIT PULL] Add EFI_CERT_X509_GUID support for dbx/mokx entries I'm guessing you're not going to pull it now for 5.12, so should I just

[PATCH v7] mm: Add set/end/wait functions for PG_private_2

2021-04-13 Thread David Howells
compound_head() in all the functions to make them THP safe[6]. v5: - Add set and end functions, calling the end function end rather than unlock[3]. - Keep a ref on the page when PG_private_2 is set[4][5]. v4: - Remove extern from the declaration[2]. Suggested-by: Linus Torvalds Signed-off-by: David

[RFC PATCH 2/2] iov_iter: Drop the X argument from iterate_all_kinds() and use B instead

2021-04-09 Thread David Howells
Drop the X argument from iterate_all_kinds() and use the B argument instead as it's always the same unless the ITER_XARRAY is handled specially. Signed-off-by: David Howells --- lib/iov_iter.c | 42 -- 1 file changed, 12 insertions(+), 30 deletions

[RFC PATCH 1/2] iov_iter: Remove iov_iter_for_each_range()

2021-04-09 Thread David Howells
Remove iov_iter_for_each_range() as it's no longer used with the removal of lustre. Signed-off-by: David Howells --- include/linux/uio.h |4 lib/iov_iter.c | 27 --- 2 files changed, 31 deletions(-) diff --git a/include/linux/uio.h b/include/linux/uio.h

Re: [RFC PATCH 2/3] mm: Return bool from pagebit test functions

2021-04-09 Thread David Howells
David Howells wrote: > add/remove: 2/2 grow/shrink: 15/16 up/down: 408/-599 (-191) > Function old new delta > iomap_write_end_inline - 128+128 I can get rid of the iomap_write_end_inline() increase for

Re: [RFC PATCH 2/3] mm: Return bool from pagebit test functions

2021-04-09 Thread David Howells
Matthew Wilcox wrote: > iirc i looked at doing this as part of the folio work, and it ended up > increasing the size of the kernel. Did you run bloat-o-meter on the > result of doing this? add/remove: 2/2 grow/shrink: 15/16 up/down: 408/-599 (-191) Function

Re: [RFC PATCH 2/3] mm: Return bool from pagebit test functions

2021-04-09 Thread David Howells
Matthew Wilcox wrote: > On Fri, Apr 09, 2021 at 11:59:17AM +0100, David Howells wrote: > > Make functions that test page bits return a bool, not an int. This means > > that the value is definitely 0 or 1 if they're used in arithmetic, rather > > than rely on test_bit()

[RFC PATCH 3/3] mm: Split page_has_private() in two to better handle PG_private_2

2021-04-09 Thread David Howells
try_to_release_page()" combos. Does it make sense to have a pot this into an inline function? Signed-off-by: David Howells cc: Linus Torvalds cc: Matthew Wilcox cc: Christoph Hellwig cc: Josef Bacik cc: Alexander Viro cc: Andrew Morton cc: linux...@kvack.org cc: linux-cach..

[RFC PATCH 2/3] mm: Return bool from pagebit test functions

2021-04-09 Thread David Howells
Make functions that test page bits return a bool, not an int. This means that the value is definitely 0 or 1 if they're used in arithmetic, rather than rely on test_bit() and friends to return this (though they probably should). Signed-off-by: David Howells cc: Linus Torvalds cc: Matthew

[RFC PATCH 1/3] Make the generic bitops return bool

2021-04-09 Thread David Howells
Make the generic bitops return bool when returning the value of a tested bit. Signed-off-by: David Howells cc: Linus Torvalds cc: Matthew Wilcox cc: Akinobu Mita cc: Arnd Bergmann cc: Will Deacon --- include/asm-generic/bitops/atomic.h |6 +++--- include/asm-generic/bitops/le.h

Re: [PATCH v6 01/30] iov_iter: Add ITER_XARRAY

2021-04-09 Thread David Howells
Al Viro wrote: > > +#define iterate_all_kinds(i, n, v, I, B, K, X) { \ > > Do you have any users that would pass different B and X? > > > @@ -1440,7 +1665,7 @@ ssize_t iov_iter_get_pages_alloc(struct iov_iter *i, > > return v.bv_len; > > }),({ > > return

Re: [RFC][PATCH] mm: Split page_has_private() in two to better handle PG_private_2

2021-04-09 Thread David Howells
Linus Torvalds wrote: > > #define PAGE_FLAGS_PRIVATE \ > > (1UL << PG_private | 1UL << PG_private_2) > > I think this should be re-named to be PAGE_FLAGS_CLEANUP, because I > don't think it makes any other sense to "combine" the two PG_private* > bits any

[RFC][PATCH] mm: Split page_has_private() in two to better handle PG_private_2

2021-04-08 Thread David Howells
unction for this? Signed-off-by: David Howells Link: https://lore.kernel.org/linux-fsdevel/CAHk-=whwojhgemn85loh9fx-5d2-upzmv1m2zmyxvd31tkp...@mail.gmail.com/ [1] --- fs/btrfs/disk-io.c |2 +- fs/btrfs/inode.c |2 +- fs/ext4/move_extent.c |

Re: [PATCH v6 02/30] mm: Add set/end/wait functions for PG_private_2

2021-04-08 Thread David Howells
: David Howells Date: Thu Apr 8 16:33:20 2021 +0100 netfs: Fix PG_private_2 helper functions to consistently use compound_head() diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index ef511364cc0c..63ca6430aef5 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h

Re: [PATCH v6 02/30] mm: Add set/end/wait functions for PG_private_2

2021-04-08 Thread David Howells
Matthew Wilcox wrote: > > +void end_page_private_2(struct page *page) > > +{ > > + page = compound_head(page); > > + VM_BUG_ON_PAGE(!PagePrivate2(page), page); > > + clear_bit_unlock(PG_private_2, >flags); > > + wake_up_page_bit(page, PG_private_2); > > ... but when we try to end on a

Re: [PATCH v2 00/18] Implement RSASSA-PSS signature verification

2021-04-08 Thread David Howells
Varad Gautam wrote: > The test harness is available at [5]. Can you add this to the keyutils testsuite? https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/keyutils.git David

Re: [PATCH v2 18/18] keyctl_pkey: Add pkey parameters slen and mgfhash for PSS

2021-04-08 Thread David Howells
Varad Gautam wrote: > + Opt_slen, /* "slen=" eg. "slen=32" */ "slen" seems a bit unobvious. Maybe "saltlen=..."? David

[PATCH v6 30/30] afs: Use the netfs_write_begin() helper

2021-04-08 Thread David Howells
: A more advanced version of this could be made that does generic_perform_write() for a whole cache granule. This would make it easier to avoid doing the download/read for the data to be overwritten. Signed-off-by: David Howells cc: linux-...@lists.infradead.org cc: linux-cach...@redhat.com cc: linux

[PATCH v6 29/30] afs: Use new netfs lib read helper API

2021-04-08 Thread David Howells
ded in error handling fixes to afs_req_issue_op(). - Added flag to netfs_subreq_terminated() to indicate that the caller may have been running async and stuff that might sleep needs punting to a workqueue. Signed-off-by: David Howells cc: linux-...@lists.infradead.org cc: linux-cach...@redhat.

[PATCH v6 28/30] afs: Use the fs operation ops to handle FetchData completion

2021-04-08 Thread David Howells
. This means the done function will be called back on error also, not just on successful completion. This allows motion towards asynchronous data reception on data fetch calls and allows any error to be handed off to the fscache read helper in the same place as a successful completion. Signed-off-by: David

[PATCH v6 27/30] afs: Prepare for use of THPs

2021-04-08 Thread David Howells
As a prelude to supporting transparent huge pages, use thp_size() and similar rather than PAGE_SIZE/SHIFT. Further, try and frame everything in terms of file positions and lengths rather than page indices and numbers of pages. Signed-off-by: David Howells cc: linux-...@lists.infradead.org cc

[PATCH v6 26/30] afs: Extract writeback extension into its own function

2021-04-08 Thread David Howells
Extract writeback extension into its own function to break up the writeback function a bit. Signed-off-by: David Howells cc: linux-...@lists.infradead.org cc: linux-cach...@redhat.com cc: linux-fsde...@vger.kernel.org Link: https://lore.kernel.org/r/160588538471.3465195.782513375683399583.st

[PATCH v6 25/30] afs: Wait on PG_fscache before modifying/releasing a page

2021-04-08 Thread David Howells
PG_fscache is going to be used to indicate that a page is being written to the cache, and that the page should not be modified or released until it's finished. Make afs_invalidatepage() and afs_releasepage() wait for it. Signed-off-by: David Howells cc: linux-...@lists.infradead.org cc: linux

[PATCH v6 24/30] afs: Use ITER_XARRAY for writing

2021-04-08 Thread David Howells
potentially allocating a huge array of bio_vec structs (max 256 per page) when in fact the pagecache already has a structure listing all the relevant pages (radix_tree/xarray) that can be walked over. Signed-off-by: David Howells cc: linux-...@lists.infradead.org cc: linux-cach...@redhat.com cc: linux

[PATCH v6 23/30] afs: Set up the iov_iter before calling afs_extract_data()

2021-04-08 Thread David Howells
. This will allow O_DIRECT calls to be supported in future patches. Signed-off-by: David Howells cc: linux-...@lists.infradead.org cc: linux-cach...@redhat.com cc: linux-fsde...@vger.kernel.org Link: https://lore.kernel.org/r/152898380012.11616.12094591785228251717.st...@warthog.procyon.org.uk/ Link

[PATCH v6 22/30] afs: Log remote unmarshalling errors

2021-04-08 Thread David Howells
Log unmarshalling errors reported by the peer (ie. it can't parse what we sent it). Limit the maximum number of messages to 3. Signed-off-by: David Howells cc: linux-...@lists.infradead.org cc: linux-cach...@redhat.com cc: linux-fsde...@vger.kernel.org Link: https://lore.kernel.org/r

[PATCH v6 21/30] afs: Don't truncate iter during data fetch

2021-04-08 Thread David Howells
the iterator. Signed-off-by: David Howells cc: linux-...@lists.infradead.org cc: linux-cach...@redhat.com cc: linux-fsde...@vger.kernel.org Link: https://lore.kernel.org/r/158861249201.340223.13035445866976590375.st...@warthog.procyon.org.uk/ # rfc Link: https://lore.kernel.org/r

[PATCH v6 20/30] afs: Move key to afs_read struct

2021-04-08 Thread David Howells
Stash the key used to authenticate read operations in the afs_read struct. This will be necessary to reissue the operation against the server if a read from the cache fails in upcoming cache changes. Signed-off-by: David Howells cc: linux-...@lists.infradead.org cc: linux-cach...@redhat.com cc

[PATCH v6 19/30] afs: Print the operation debug_id when logging an unexpected data version

2021-04-08 Thread David Howells
Print the afs_operation debug_id when logging an unexpected change in the data version. This allows the logged message to be matched against tracelines. Signed-off-by: David Howells cc: linux-...@lists.infradead.org cc: linux-cach...@redhat.com cc: linux-fsde...@vger.kernel.org Link: https

[PATCH v6 18/30] afs: Pass page into dirty region helpers to provide THP size

2021-04-08 Thread David Howells
or private data as these can be retrieved directly from the page struct. Signed-off-by: David Howells cc: linux-...@lists.infradead.org cc: linux-cach...@redhat.com cc: linux-fsde...@vger.kernel.org Link: https://lore.kernel.org/r/160588527183.3465195.16107942526481976308.st...@warthog.procyon.org.uk

[PATCH v6 15/30] netfs: Add a tracepoint to log failures that would be otherwise unseen

2021-04-08 Thread David Howells
Add a tracepoint to log internal failures (such as cache errors) that we don't otherwise want to pass back to the netfs. Signed-off-by: David Howells cc: Matthew Wilcox cc: linux...@kvack.org cc: linux-cach...@redhat.com cc: linux-...@lists.infradead.org cc: linux-...@vger.kernel.org cc: linux

[PATCH v6 17/30] afs: Disable use of the fscache I/O routines

2021-04-08 Thread David Howells
Disable use of the fscache I/O routined by the AFS filesystem. It's about to transition to passing iov_iters down and fscache is about to have its I/O path to use iov_iter, so all that needs to change. Signed-off-by: David Howells cc: linux-...@lists.infradead.org cc: linux-cach...@redhat.com

[PATCH v6 16/30] fscache, cachefiles: Add alternate API to use kiocb for read/write to cache

2021-04-08 Thread David Howells
)[2]. - Remove explicit NULL initialiser[2]. - Remove extern on func decl[2]. - Put in param names on func decl[2]. - Remove redundant else[2]. - Fill out the kdoc comment for fscache_begin_read_operation(). - Rename fs/fscache/page2.c to io.c to match later patches. Signed-off-by: David Howells Revie

[PATCH v6 14/30] netfs: Define an interface to talk to a cache

2021-04-08 Thread David Howells
eep needs punting to a workqueue (can't use in_softirq()[1]). - Add missing inc of netfs_n_rh_read stat. - Move initial definition of fscache_begin_read_operation() elsewhere. - Need to call op->begin_cache_operation() from netfs_write_begin(). Signed-off-by: David Howells Reviewed-by: Jeff

Re: [PATCH] afs: fix no return statement in function returning non-void

2021-04-08 Thread David Howells
Zheng Zengkai wrote: > static int afs_dir_set_page_dirty(struct page *page) > { > BUG(); /* This should never happen. */ > + return 0; > } That shouldn't be necessary. BUG() should be marked as 'no return' to the compiler. What arch and compiler are you using? David

[PATCH v6 13/30] netfs: Add write_begin helper

2021-04-08 Thread David Howells
() to indicate that the caller may have been running async and stuff that might sleep needs punting to a workqueue (can't use in_softirq()[1]). Signed-off-by: David Howells Reviewed-by: Jeff Layton cc: Matthew Wilcox cc: linux...@kvack.org cc: linux-cach...@redhat.com cc: linux

[PATCH v6 12/30] netfs: Gather stats

2021-04-08 Thread David Howells
Gather statistics from the netfs interface that can be exported through a seqfile. This is intended to be called by a later patch when viewing /proc/fs/fscache/stats. Signed-off-by: David Howells Reviewed-by: Jeff Layton cc: Matthew Wilcox cc: linux...@kvack.org cc: linux-cach...@redhat.com

[PATCH v6 11/30] netfs: Add tracepoints

2021-04-08 Thread David Howells
requests. A read request may be a compound of multiple subrequests. (3) netfs/netfs_sreq This logs the progress of netfs_read_subrequest objects, which track the contributions from various sources to a read request. Signed-off-by: David Howells Reviewed-by: Jeff Layton cc: Matthew

[PATCH v6 10/30] netfs: Provide readahead and readpage netfs helpers

2021-04-08 Thread David Howells
nning async and stuff that might sleep needs punting to a workqueue (can't use in_softirq()[1]). Signed-off-by: David Howells Reviewed-by: Jeff Layton cc: Matthew Wilcox cc: linux...@kvack.org cc: linux-cach...@redhat.com cc: linux-...@lists.infradead.org cc: linux-...@vger.kernel.org cc: linux

[PATCH v6 09/30] netfs, mm: Add set/end/wait_on_page_fscache() aliases

2021-04-08 Thread David Howells
functions into the caching-specific headers rather than pagemap.h[1]] Changes: v5: - Mirror the changes to the core routines[2]. Signed-off-by: David Howells cc: Linus Torvalds cc: Matthew Wilcox cc: linux...@kvack.org cc: linux-cach...@redhat.com cc: linux-...@lists.infradead.org cc: linux

[PATCH v6 08/30] netfs, mm: Move PG_fscache helper funcs to linux/netfs.h

2021-04-08 Thread David Howells
Move the PG_fscache related helper funcs (such as SetPageFsCache()) to linux/netfs.h rather than linux/fscache.h as the intention is to move to a model where they're used by the network filesystem and the helper library, but not by fscache/cachefiles itself. Signed-off-by: David Howells cc

[PATCH v6 07/30] netfs: Documentation for helper library

2021-04-08 Thread David Howells
Add interface documentation for the netfs helper library. Signed-off-by: David Howells cc: linux...@kvack.org cc: linux-cach...@redhat.com cc: linux-...@lists.infradead.org cc: linux-...@vger.kernel.org cc: linux-c...@vger.kernel.org cc: ceph-de...@vger.kernel.org cc: v9fs-develo

[PATCH v6 06/30] netfs: Make a netfs helper module

2021-04-08 Thread David Howells
Make a netfs helper module to manage read request segmentation, caching support and transparent huge page support on behalf of a network filesystem. Signed-off-by: David Howells Reviewed-by: Jeff Layton cc: Matthew Wilcox cc: linux...@kvack.org cc: linux-cach...@redhat.com cc: linux

[PATCH v6 04/30] fs: Document file_ra_state

2021-04-08 Thread David Howells
From: Matthew Wilcox (Oracle) Turn the comments into kernel-doc and improve the wording slightly. Signed-off-by: Matthew Wilcox (Oracle) Signed-off-by: David Howells Link: https://lore.kernel.org/r/20210407201857.3582797-3-wi...@infradead.org/ --- include/linux/fs.h | 24

[PATCH v6 05/30] mm: Implement readahead_control pageset expansion

2021-04-08 Thread David Howells
the declaration of readahead_expand() to a better place[1]. Suggested-by: Matthew Wilcox (Oracle) Signed-off-by: David Howells cc: Matthew Wilcox (Oracle) cc: Alexander Viro cc: Christoph Hellwig cc: Mike Marshall cc: linux...@kvack.org cc: linux-cach...@redhat.com cc: linux-...@lists.infradead.org cc

[PATCH v6 03/30] mm/filemap: Pass the file_ra_state in the ractl

2021-04-08 Thread David Howells
-by: David Howells Link: https://lore.kernel.org/r/20210407201857.3582797-2-wi...@infradead.org/ --- fs/ext4/verity.c|2 +- fs/f2fs/file.c |2 +- fs/f2fs/verity.c|2 +- include/linux/pagemap.h | 20 +++- mm/filemap.c|4 ++-- mm

[PATCH v6 02/30] mm: Add set/end/wait functions for PG_private_2

2021-04-08 Thread David Howells
and end functions, calling the end function end rather than unlock[3]. - Keep a ref on the page when PG_private_2 is set[4][5]. v4: - Remove extern from the declaration[2]. Suggested-by: Linus Torvalds Signed-off-by: David Howells cc: Matthew Wilcox (Oracle) cc: Alexander Viro cc: Christoph

[PATCH v6 01/30] iov_iter: Add ITER_XARRAY

2021-04-08 Thread David Howells
, that would require a bio_vec array to be allocated to refer to all the pages - which should be redundant if inode->i_pages also points to all these pages. Note that older versions of this patch implemented an ITER_MAPPING instead, which was almost the same. Signed-off-by: David Howells cc: Alexander V

[PATCH v6 00/30] Network fs helper library & fscache kiocb API

2021-04-08 Thread David Howells
rnel.org/r/270998.1612265...@warthog.procyon.org.uk/ David --- David Howells (28): iov_iter: Add ITER_XARRAY mm: Add set/end/wait functions for PG_private_2 mm: Implement readahead_control pageset expansion netfs: Make a netfs helper module netfs: Documentation for help

[PATCH 5/5] netfs: Add a tracepoint to log failures that would be otherwise unseen

2021-04-07 Thread David Howells
Add a tracepoint to log internal failures (such as cache errors) that we don't otherwise want to pass back to the netfs. Signed-off-by: David Howells --- fs/netfs/read_helper.c | 14 +- include/trace/events/netfs.h | 58 ++ 2 files

[PATCH 4/5] netfs: Fix copy-to-cache amalgamation

2021-04-07 Thread David Howells
. Instead, we only need to deal with contiguous subreqs and then ask the cache to round off as it needs - which also means we don't have to make any assumptions about the cache granularity. Signed-off-by: David Howells --- fs/cachefiles/io.c | 17 + fs/netfs

[PATCH 3/5] netfs: Don't record the copy termination error

2021-04-07 Thread David Howells
Don't record the copy termination error in the subrequest. We shouldn't return it through netfs_readpage() or netfs_write_begin() as we don't let the netfs see cache errors. Signed-off-by: David Howells --- fs/netfs/read_helper.c |2 -- 1 file changed, 2 deletions(-) diff --git a/fs

[PATCH 2/5] netfs: Call trace_netfs_read() after ->begin_cache_operation()

2021-04-07 Thread David Howells
Reorder the netfs library API functions slightly to log the netfs_read tracepoint after calling out to the network filesystem to begin a caching operation. This sets rreq->cookie_debug_id so that it can be logged in tracepoints. Signed-off-by: David Howells --- fs/netfs/read_helper.c |

[PATCH 1/5] netfs: Fix a missing rreq put in netfs_write_begin()

2021-04-07 Thread David Howells
netfs_write_begin() needs to drop a ref on the read request if the network filesystem gives an error when called to begin the caching op. Signed-off-by: David Howells --- fs/netfs/read_helper.c |2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/netfs/read_helper.c b/fs/netfs

[PATCH 0/5] netfs: Fixes for the netfs lib

2021-04-07 Thread David Howells
Hi Jeff, Here's a bunch of fixes plus a tracepoint for the netfs library. I'm going to roll them into other patches, but I'm posting them here for separate review. David --- David Howells (5): netfs: Fix a missing rreq put in netfs_write_begin() netfs: Call trace_netfs_read

Re: [PATCH] net/rxrpc: Fix a use after free in rxrpc_input_packet

2021-04-01 Thread David Howells
Lv Yunlong wrote: > In the case RXRPC_PACKET_TYPE_DATA of rxrpc_input_packet, if > skb_unshare(skb,..) failed, it will free the skb and return NULL. > But if skb_unshare() return NULL, the freed skb will be used by > rxrpc_eaten_skb(skb,..). That's not precisely the case: void

Re: [PATCH v1 0/3] KEYS: trusted: Introduce support for NXP CAAM-based trusted keys

2021-04-01 Thread David Howells
Richard Weinberger wrote: > On Wed, Mar 17, 2021 at 3:08 PM Ahmad Fatoum wrote: > > keyctl add trusted $KEYNAME "load $(cat ~/kmk.blob)" @s > > Is there a reason why we can't pass the desired backend name in the > trusted key parameters? > e.g. > keyctl add trusted $KEYNAME "backendtype

[PATCH] cachefiles: do not yet allow on idmapped mounts

2021-03-24 Thread David Howells
the files wouldn't match to what they expressed in the idmapping. Block cache files on idmapped mounts until the fscache rework is done and we have ported it to support idmapped mounts. Signed-off-by: Christian Brauner Signed-off-by: David Howells Cc: linux-cach...@redhat.com Link: https://lore.

[GIT PULL] cachefiles, afs: mm wait fixes

2021-03-24 Thread David Howells
Hi Linus, Could you pull these patches from Matthew Wilcox to fix page waiting-related issues in cachefiles and afs as extracted from his folio series[1]: (1) In cachefiles, remove the use of the wait_bit_key struct to access something that's actually in wait_page_key format. The proper

[PATCH v5 28/28] afs: Use the fscache_write_begin() helper

2021-03-23 Thread David Howells
: A more advanced version of this could be made that does generic_perform_write() for a whole cache granule. This would make it easier to avoid doing the download/read for the data to be overwritten. Signed-off-by: David Howells cc: linux-...@lists.infradead.org cc: linux-cach...@redhat.com cc: linux

[PATCH v5 27/28] afs: Use new fscache read helper API

2021-03-23 Thread David Howells
v4: - Folded in error handling fixes to afs_req_issue_op(). - Added flag to netfs_subreq_terminated() to indicate that the caller may have been running async and stuff that might sleep needs punting to a workqueue. Signed-off-by: David Howells cc: linux-...@lists.infradead.org cc: linux-c

[PATCH v5 26/28] afs: Use the fs operation ops to handle FetchData completion

2021-03-23 Thread David Howells
. This means the done function will be called back on error also, not just on successful completion. This allows motion towards asynchronous data reception on data fetch calls and allows any error to be handed off to the fscache read helper in the same place as a successful completion. Signed-off-by: David

[PATCH v5 25/28] afs: Prepare for use of THPs

2021-03-23 Thread David Howells
As a prelude to supporting transparent huge pages, use thp_size() and similar rather than PAGE_SIZE/SHIFT. Further, try and frame everything in terms of file positions and lengths rather than page indices and numbers of pages. Signed-off-by: David Howells cc: linux-...@lists.infradead.org cc

[PATCH v5 24/28] afs: Extract writeback extension into its own function

2021-03-23 Thread David Howells
Extract writeback extension into its own function to break up the writeback function a bit. Signed-off-by: David Howells cc: linux-...@lists.infradead.org cc: linux-cach...@redhat.com cc: linux-fsde...@vger.kernel.org Link: https://lore.kernel.org/r/160588538471.3465195.782513375683399583.st

[PATCH v5 23/28] afs: Wait on PG_fscache before modifying/releasing a page

2021-03-23 Thread David Howells
PG_fscache is going to be used to indicate that a page is being written to the cache, and that the page should not be modified or released until it's finished. Make afs_invalidatepage() and afs_releasepage() wait for it. Signed-off-by: David Howells cc: linux-...@lists.infradead.org cc: linux

[PATCH v5 22/28] afs: Use ITER_XARRAY for writing

2021-03-23 Thread David Howells
potentially allocating a huge array of bio_vec structs (max 256 per page) when in fact the pagecache already has a structure listing all the relevant pages (radix_tree/xarray) that can be walked over. Signed-off-by: David Howells cc: linux-...@lists.infradead.org cc: linux-cach...@redhat.com cc: linux

[PATCH v5 21/28] afs: Set up the iov_iter before calling afs_extract_data()

2021-03-23 Thread David Howells
. This will allow O_DIRECT calls to be supported in future patches. Signed-off-by: David Howells cc: linux-...@lists.infradead.org cc: linux-cach...@redhat.com cc: linux-fsde...@vger.kernel.org Link: https://lore.kernel.org/r/152898380012.11616.12094591785228251717.st...@warthog.procyon.org.uk/ Link

[PATCH v5 20/28] afs: Log remote unmarshalling errors

2021-03-23 Thread David Howells
Log unmarshalling errors reported by the peer (ie. it can't parse what we sent it). Limit the maximum number of messages to 3. Signed-off-by: David Howells cc: linux-...@lists.infradead.org cc: linux-cach...@redhat.com cc: linux-fsde...@vger.kernel.org Link: https://lore.kernel.org/r

[PATCH v5 19/28] afs: Don't truncate iter during data fetch

2021-03-23 Thread David Howells
the iterator. Signed-off-by: David Howells cc: linux-...@lists.infradead.org cc: linux-cach...@redhat.com cc: linux-fsde...@vger.kernel.org Link: https://lore.kernel.org/r/158861249201.340223.13035445866976590375.st...@warthog.procyon.org.uk/ # rfc Link: https://lore.kernel.org/r

[PATCH v5 17/28] afs: Print the operation debug_id when logging an unexpected data version

2021-03-23 Thread David Howells
Print the afs_operation debug_id when logging an unexpected change in the data version. This allows the logged message to be matched against tracelines. Signed-off-by: David Howells cc: linux-...@lists.infradead.org cc: linux-cach...@redhat.com cc: linux-fsde...@vger.kernel.org Link: https

[PATCH v5 18/28] afs: Move key to afs_read struct

2021-03-23 Thread David Howells
Stash the key used to authenticate read operations in the afs_read struct. This will be necessary to reissue the operation against the server if a read from the cache fails in upcoming cache changes. Signed-off-by: David Howells cc: linux-...@lists.infradead.org cc: linux-cach...@redhat.com cc

[PATCH v5 16/28] afs: Pass page into dirty region helpers to provide THP size

2021-03-23 Thread David Howells
or private data as these can be retrieved directly from the page struct. Signed-off-by: David Howells cc: linux-...@lists.infradead.org cc: linux-cach...@redhat.com cc: linux-fsde...@vger.kernel.org Link: https://lore.kernel.org/r/160588527183.3465195.16107942526481976308.st...@warthog.procyon.org.uk

[PATCH v5 15/28] afs: Disable use of the fscache I/O routines

2021-03-23 Thread David Howells
Disable use of the fscache I/O routined by the AFS filesystem. It's about to transition to passing iov_iters down and fscache is about to have its I/O path to use iov_iter, so all that needs to change. Signed-off-by: David Howells cc: linux-...@lists.infradead.org cc: linux-cach...@redhat.com

[PATCH v5 14/28] fscache, cachefiles: Add alternate API to use kiocb for read/write to cache

2021-03-23 Thread David Howells
l[2]. - Remove redundant else[2]. - Fill out the kdoc comment for fscache_begin_read_operation(). - Rename fs/fscache/page2.c to io.c to match later patches. Signed-off-by: David Howells Reviewed-by: Jeff Layton cc: Christoph Hellwig cc: linux-cach...@redhat.com cc: linux-...@lists.infradead.org

[PATCH v5 13/28] netfs: Define an interface to talk to a cache

2021-03-23 Thread David Howells
use in_softirq()[1]). - Add missing inc of netfs_n_rh_read stat. - Move initial definition of fscache_begin_read_operation() elsewhere. - Need to call op->begin_cache_operation() from netfs_write_begin(). Signed-off-by: David Howells Reviewed-by: Jeff Layton cc: Matthew Wilcox cc:

[PATCH v5 12/28] netfs: Add write_begin helper

2021-03-23 Thread David Howells
needs punting to a workqueue (can't use in_softirq()[1]). Signed-off-by: David Howells Reviewed-by: Jeff Layton cc: Matthew Wilcox cc: linux...@kvack.org cc: linux-cach...@redhat.com cc: linux-...@lists.infradead.org cc: linux-...@vger.kernel.org cc: linux-c...@vger.kernel.org cc: ceph-de

[PATCH v5 11/28] netfs: Gather stats

2021-03-23 Thread David Howells
Gather statistics from the netfs interface that can be exported through a seqfile. This is intended to be called by a later patch when viewing /proc/fs/fscache/stats. Signed-off-by: David Howells Reviewed-by: Jeff Layton cc: Matthew Wilcox cc: linux...@kvack.org cc: linux-cach...@redhat.com

[PATCH v5 10/28] netfs: Add tracepoints

2021-03-23 Thread David Howells
requests. A read request may be a compound of multiple subrequests. (3) netfs/netfs_sreq This logs the progress of netfs_read_subrequest objects, which track the contributions from various sources to a read request. Signed-off-by: David Howells Reviewed-by: Jeff Layton cc: Matthew

[PATCH v5 09/28] netfs: Provide readahead and readpage netfs helpers

2021-03-23 Thread David Howells
nning async and stuff that might sleep needs punting to a workqueue (can't use in_softirq()[1]). Signed-off-by: David Howells Reviewed-by: Jeff Layton cc: Matthew Wilcox cc: linux...@kvack.org cc: linux-cach...@redhat.com cc: linux-...@lists.infradead.org cc: linux-...@vger.kernel.org cc: linux

[PATCH v5 08/28] netfs, mm: Add set/end/wait_on_page_fscache() aliases

2021-03-23 Thread David Howells
functions into the caching-specific headers rather than pagemap.h[1]] Changes: v5: - Mirror the changes to the core routines[2]. Signed-off-by: David Howells cc: Linus Torvalds cc: Matthew Wilcox cc: linux...@kvack.org cc: linux-cach...@redhat.com cc: linux-...@lists.infradead.org cc: linux

[PATCH v5 07/28] netfs, mm: Move PG_fscache helper funcs to linux/netfs.h

2021-03-23 Thread David Howells
Move the PG_fscache related helper funcs (such as SetPageFsCache()) to linux/netfs.h rather than linux/fscache.h as the intention is to move to a model where they're used by the network filesystem and the helper library, but not by fscache/cachefiles itself. Signed-off-by: David Howells cc

[PATCH v5 06/28] netfs: Documentation for helper library

2021-03-23 Thread David Howells
Add interface documentation for the netfs helper library. Signed-off-by: David Howells cc: linux...@kvack.org cc: linux-cach...@redhat.com cc: linux-...@lists.infradead.org cc: linux-...@vger.kernel.org cc: linux-c...@vger.kernel.org cc: ceph-de...@vger.kernel.org cc: v9fs-develo

[PATCH v5 05/28] netfs: Make a netfs helper module

2021-03-23 Thread David Howells
Make a netfs helper module to manage read request segmentation, caching support and transparent huge page support on behalf of a network filesystem. Signed-off-by: David Howells Reviewed-by: Jeff Layton cc: Matthew Wilcox cc: linux...@kvack.org cc: linux-cach...@redhat.com cc: linux

[PATCH v5 04/28] mm: Implement readahead_control pageset expansion

2021-03-23 Thread David Howells
boundaries, thereby allowing whole granules to be stored in the cache, but there are other potential users also. Changes: - Moved the declaration of readahead_expand() to a better place[1]. Suggested-by: Matthew Wilcox (Oracle) Signed-off-by: David Howells cc: Matthew Wilcox (Oracle) cc: Alexander Viro

[PATCH v5 03/28] mm: Add set/end/wait functions for PG_private_2

2021-03-23 Thread David Howells
and end functions, calling the end function end rather than unlock[3]. - Keep a ref on the page when PG_private_2 is set[4][5]. v4: - Remove extern from the declaration[2]. Suggested-by: Linus Torvalds Signed-off-by: David Howells cc: Matthew Wilcox (Oracle) cc: Alexander Viro cc: Christoph

[PATCH v5 02/28] mm: Add wait_on_page_writeback_killable()

2021-03-23 Thread David Howells
Add a function to wait killably on the PG_writeback page flag. Signed-off-by: David Howells cc: Matthew Wilcox (Oracle) cc: Alexander Viro cc: Christoph Hellwig cc: linux...@kvack.org cc: linux-cach...@redhat.com cc: linux-...@lists.infradead.org cc: linux-...@vger.kernel.org cc: linux-c

[PATCH v5 01/28] iov_iter: Add ITER_XARRAY

2021-03-23 Thread David Howells
, that would require a bio_vec array to be allocated to refer to all the pages - which should be redundant if inode->i_pages also points to all these pages. Note that older versions of this patch implemented an ITER_MAPPING instead, which was almost the same. Signed-off-by: David Howells cc: Alexander V

[PATCH v5 00/28] Network fs helper library & fscache kiocb API

2021-03-23 Thread David Howells
rnel.org/r/1794123.1605713...@warthog.procyon.org.uk/ Link: https://lore.kernel.org/r/241017.1612263...@warthog.procyon.org.uk/ Link: https://lore.kernel.org/r/270998.1612265...@warthog.procyon.org.uk/ David --- David Howells (28): iov_iter: Add ITER_XARRAY mm: Add wait_on_page_writebac

Re: [PATCH v4 02/28] mm: Add an unlock function for PG_private_2/PG_fscache

2021-03-23 Thread David Howells
David Howells wrote: > > > - wait_on_page_writeback(page); > > > + if (wait_on_page_writeback_killable(page) < 0) > > > + return VM_FAULT_RETRY | VM_FAULT_LOCKED; > > > > You forgot to unlock the page. > > Do I need to? Doe

Re: [PATCH v5 00/27] Memory Folios

2021-03-23 Thread David Howells
Johannes Weiner wrote: > So I fully agree with the motivation behind this patch. But I do > wonder why it's special-casing the commmon case instead of the rare > case. It comes at a huge cost. Short term, the churn of replacing > 'page' with 'folio' in pretty much all instances is enormous. > >

Re: [PATCH v4 02/28] mm: Add an unlock function for PG_private_2/PG_fscache

2021-03-23 Thread David Howells
Matthew Wilcox wrote: > On Tue, Mar 23, 2021 at 01:17:20PM +0000, David Howells wrote: > > +++ b/fs/afs/write.c > > @@ -846,7 +846,7 @@ vm_fault_t afs_page_mkwrite(struct vm_fault *vmf) > > */ > > #ifdef CONFIG_AFS_FSCACHE &

Re: [PATCH v4 02/28] mm: Add an unlock function for PG_private_2/PG_fscache

2021-03-23 Thread David Howells
David Howells wrote: > Matthew Wilcox wrote: > > > That also brings up that there is no set_page_private_2(). I think > > that's OK -- you only set PageFsCache() immediately after reading the > > page from the server. But I feel this "unloc

[PATCH 3/3] afs: Use wait_on_page_writeback_killable

2021-03-23 Thread David Howells
xes: 1cf7a1518aef ("afs: Implement shared-writeable mmap") Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Christoph Hellwig Signed-off-by: David Howells cc: linux-...@lists.infradead.org cc: linux...@kvack.org Link: https://lore.kernel.org/r/20210320054104.1300774-4-wi...@infradead

[PATCH 2/3] mm/writeback: Add wait_on_page_writeback_killable

2021-03-23 Thread David Howells
From: Matthew Wilcox (Oracle) This is the killable version of wait_on_page_writeback. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Christoph Hellwig Signed-off-by: David Howells cc: linux-...@lists.infradead.org cc: linux...@kvack.org Link: https://lore.kernel.org/r

[PATCH 1/3] fs/cachefiles: Remove wait_bit_key layout dependency

2021-03-23 Thread David Howells
by Linus[1] and that is no longer applicable. Fixes: 62906027091f ("mm: add PageWaiters indicating tasks are waiting for a page bit") Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Christoph Hellwig Signed-off-by: David Howells cc: linux-cach...@redhat.com cc: linux...@kvack.org L

[PATCH 0/3] cachefiles, afs: mm wait fixes

2021-03-23 Thread David Howells
Here are some patches to fix page waiting-related issues in cachefiles and afs[1]: (1) In cachefiles, remove the use of the wait_bit_key struct to access something that's actually in wait_page_key format. The proper struct is now available in the header, so that should be used

Re: [PATCH v5 03/27] afs: Use wait_on_page_writeback_killable

2021-03-23 Thread David Howells
Matthew Wilcox (Oracle) wrote: > Open-coding this function meant it missed out on the recent bugfix Would that be: c2407cf7d22d0c0d94cf20342b3b8f06f1d904e7 mm: make wait_on_page_writeback() wait for multiple pending writebacks David

  1   2   3   4   5   6   7   8   9   10   >