On Thu, 4 Jun 2026 14:09:55 +0530 "Aneesh Kumar K.V (Arm)" <[email protected]> wrote:
> swiotlb_dyn_free() is used after removing a dynamic swiotlb pool from > RCU-protected lists. It can call swiotlb_free_tlb(), which may need to > restore the encryption state of an unencrypted pool with > set_memory_encrypted() before freeing the pages. > > RCU callbacks run in atomic context, but set_memory_encrypted() is not > guaranteed to be atomic-safe on all architectures. For example, page > attribute updates may allocate page tables or take sleeping locks. Good catch! > Use queue_rcu_work() for dynamic pool freeing instead. This keeps the RCU > grace period before freeing a published pool, while running the actual pool > teardown from workqueue context. Use the same helper for the transient-pool > error path, since that path may also be reached from atomic DMA mapping > context. Strictly speaking, it's not necessary, because this is in the error path just after allocating a transient pool. There are only two possible scenarios: a. The transient buffer was allocated from a sleeping context, and then it's also OK to decrypt memory. b. The transient buffer was allocated in atomic context, but then it was allocated from a coherent pool and it is returned to that pool rather than decrypted. However, it's also fine to queue an RCU work. The logic is definitely cleaner and easier to maintain. > Tested-by: Michael Kelley <[email protected]> > Tested-by: Mostafa Saleh <[email protected]> > Signed-off-by: Aneesh Kumar K.V (Arm) <[email protected]> Reviewed-by: Petr Tesarik <[email protected]> Petr T > --- > include/linux/swiotlb.h | 4 ++-- > kernel/dma/swiotlb.c | 19 +++++++++++-------- > 2 files changed, 13 insertions(+), 10 deletions(-) > > diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h > index 4dcbf3931be1..526f82e9da45 100644 > --- a/include/linux/swiotlb.h > +++ b/include/linux/swiotlb.h > @@ -64,7 +64,7 @@ extern void __init swiotlb_update_mem_attributes(void); > * @areas: Array of memory area descriptors. > * @slots: Array of slot descriptors. > * @node: Member of the IO TLB memory pool list. > - * @rcu: RCU head for swiotlb_dyn_free(). > + * @dyn_free: RCU work item used to free the pool from process > context. > * @transient: %true if transient memory pool. > */ > struct io_tlb_pool { > @@ -79,7 +79,7 @@ struct io_tlb_pool { > struct io_tlb_slot *slots; > #ifdef CONFIG_SWIOTLB_DYNAMIC > struct list_head node; > - struct rcu_head rcu; > + struct rcu_work dyn_free; > bool transient; > bool unencrypted; > #endif > diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c > index f4e8b241a1c4..4c56f64602ea 100644 > --- a/kernel/dma/swiotlb.c > +++ b/kernel/dma/swiotlb.c > @@ -774,13 +774,10 @@ static void swiotlb_dyn_alloc(struct work_struct *work) > add_mem_pool(mem, pool); > } > > -/** > - * swiotlb_dyn_free() - RCU callback to free a memory pool > - * @rcu: RCU head in the corresponding struct io_tlb_pool. > - */ > -static void swiotlb_dyn_free(struct rcu_head *rcu) > +static void swiotlb_dyn_free_work(struct work_struct *work) > { > - struct io_tlb_pool *pool = container_of(rcu, struct io_tlb_pool, rcu); > + struct io_tlb_pool *pool = > + container_of(to_rcu_work(work), struct io_tlb_pool, dyn_free); > size_t slots_size = array_size(sizeof(*pool->slots), pool->nslabs); > size_t tlb_size = pool->end - pool->start; > > @@ -789,6 +786,12 @@ static void swiotlb_dyn_free(struct rcu_head *rcu) > kfree(pool); > } > > +static void swiotlb_schedule_dyn_free(struct io_tlb_pool *pool) > +{ > + INIT_RCU_WORK(&pool->dyn_free, swiotlb_dyn_free_work); > + queue_rcu_work(system_wq, &pool->dyn_free); > +} > + > /** > * __swiotlb_find_pool() - find the IO TLB pool for a physical address > * @dev: Device which has mapped the DMA buffer. > @@ -835,7 +838,7 @@ static void swiotlb_del_pool(struct device *dev, struct > io_tlb_pool *pool) > list_del_rcu(&pool->node); > spin_unlock_irqrestore(&dev->dma_io_tlb_lock, flags); > > - call_rcu(&pool->rcu, swiotlb_dyn_free); > + swiotlb_schedule_dyn_free(pool); > } > > #endif /* CONFIG_SWIOTLB_DYNAMIC */ > @@ -1276,7 +1279,7 @@ static int swiotlb_find_slots(struct device *dev, > phys_addr_t orig_addr, > index = swiotlb_search_pool_area(dev, pool, 0, orig_addr, tbl_dma_addr, > alloc_size, alloc_align_mask); > if (index < 0) { > - swiotlb_dyn_free(&pool->rcu); > + swiotlb_schedule_dyn_free(pool); > return -1; > } >
