Private node folios should not be longterm-pinnable by default. A pinned folio is frozen in place, no migration, compaction, or reclaim, so the service loses control for the duration of the pin.
Some services may depend on hot-unplugability and must disallow longterm pinning. Others (accelerators with shared CPU-device state) need pinning to work. Add NP_OPS_LONGTERM_PIN flag for services to opt in with. Hook into folio_is_longterm_pinnable() in mm.h, which all GUP callers out-of-line helper, node_private_allows_longterm_pin(), called only for N_MEMORY_PRIVATE nodes. Without the flag: folio_is_longterm_pinnable() returns false, migration fails (no __GFP_PRIVATE in GFP mask) and pin_user_pages(FOLL_LONGTERM) returns -ENOMEM. With the flag: pin succeeds and the folio stays on the private node. Signed-off-by: Gregory Price <[email protected]> --- drivers/base/node.c | 15 +++++++++++++++ include/linux/mm.h | 22 ++++++++++++++++++++++ include/linux/node_private.h | 2 ++ 3 files changed, 39 insertions(+) diff --git a/drivers/base/node.c b/drivers/base/node.c index da523aca18fa..5d2487fd54f4 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -866,6 +866,21 @@ void register_memory_blocks_under_node_hotplug(int nid, unsigned long start_pfn, static DEFINE_MUTEX(node_private_lock); static bool node_private_initialized; +/** + * node_private_allows_longterm_pin - Check if a private node allows longterm pinning + * @nid: Node identifier + * + * Out-of-line helper for folio_is_longterm_pinnable() since mm.h cannot + * include node_private.h (circular dependency). + * + * Returns true if the node has NP_OPS_LONGTERM_PIN set. + */ +bool node_private_allows_longterm_pin(int nid) +{ + return node_private_has_flag(nid, NP_OPS_LONGTERM_PIN); +} +EXPORT_SYMBOL_GPL(node_private_allows_longterm_pin); + /** * node_private_register - Register a private node * @nid: Node identifier diff --git a/include/linux/mm.h b/include/linux/mm.h index fb1819ad42c3..9088fd08aeb9 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2192,6 +2192,13 @@ static inline bool is_zero_folio(const struct folio *folio) /* MIGRATE_CMA and ZONE_MOVABLE do not allow pin folios */ #ifdef CONFIG_MIGRATION + +#ifdef CONFIG_NUMA +bool node_private_allows_longterm_pin(int nid); +#else +static inline bool node_private_allows_longterm_pin(int nid) { return false; } +#endif + static inline bool folio_is_longterm_pinnable(struct folio *folio) { #ifdef CONFIG_CMA @@ -2215,6 +2222,21 @@ static inline bool folio_is_longterm_pinnable(struct folio *folio) if (folio_is_fsdax(folio)) return false; + /* + * Private node folios are not longterm pinnable by default. + * Services that support pinning opt in via NP_OPS_LONGTERM_PIN. + * node_private_allows_longterm_pin() is out-of-line because + * node_private.h includes mm.h (circular dependency). + * + * Guarded by CONFIG_NUMA because on !CONFIG_NUMA the single-node + * node_state() stub returns true for node 0, which would make + * all folios non-pinnable via the false-returning stub. + */ +#ifdef CONFIG_NUMA + if (node_state(folio_nid(folio), N_MEMORY_PRIVATE)) + return node_private_allows_longterm_pin(folio_nid(folio)); +#endif + /* Otherwise, non-movable zone folios can be pinned. */ return !folio_is_zone_movable(folio); diff --git a/include/linux/node_private.h b/include/linux/node_private.h index fe0336773ddb..7a7438fb9eda 100644 --- a/include/linux/node_private.h +++ b/include/linux/node_private.h @@ -144,6 +144,8 @@ struct node_private_ops { #define NP_OPS_NUMA_BALANCING BIT(5) /* Allow compaction to run on the node. Service must start kcompactd. */ #define NP_OPS_COMPACTION BIT(6) +/* Allow longterm DMA pinning (RDMA, VFIO, etc.) of folios on this node */ +#define NP_OPS_LONGTERM_PIN BIT(7) /* Private node is OOM-eligible: reclaim can run and pages can be demoted here */ #define NP_OPS_OOM_ELIGIBLE (NP_OPS_RECLAIM | NP_OPS_DEMOTION) -- 2.53.0
