[RFC PATCH 7/7] RDMA/rxe: Add support for the traditional Atomic operations with ODP

2022-09-06 Thread Daisuke Matsuda
Enable 'fetch and add' and 'compare and swap' operations to manipulate data in an ODP-enabled MR. This is comprised of the following steps: 1. Check the driver page table(umem_odp->dma_list) to see if the target page is both readable and writable. 2. If not, then trigger page fault to map the

[RFC PATCH 6/7] RDMA/rxe: Add support for Send/Recv/Write/Read operations with ODP

2022-09-06 Thread Daisuke Matsuda
rxe_mr_copy() is used widely to copy data to/from a user MR. requester uses it to load payloads of requesting packets; responder uses it to process Send, Write, and Read operaetions; completer uses it to copy data from response packets of Read and Atomic operations to a user MR. Allow these operat

[RFC PATCH 5/7] RDMA/rxe: Allow registering MRs for On-Demand Paging

2022-09-06 Thread Daisuke Matsuda
Allow applications to register an ODP-enabled MR, in which case the flag IB_ACCESS_ON_DEMAND is passed to rxe_reg_user_mr(). However, there is no RDMA operation supported right now. They will be enabled later in the subsequent two patches. rxe_odp_do_pagefault() is called to initialize an ODP-enab

[RFC PATCH 3/7] RDMA/rxe: Cleanup code for responder Atomic operations

2022-09-06 Thread Daisuke Matsuda
Currently, rxe_responder() directly calls the function to execute Atomic operations. This need to be modified to insert some conditional branches for the new RDMA Write operation and the ODP feature. Signed-off-by: Daisuke Matsuda --- drivers/infiniband/sw/rxe/rxe_resp.c | 102 +-

[RFC PATCH 2/7] RDMA/rxe: Convert the triple tasklets to workqueues

2022-09-06 Thread Daisuke Matsuda
In order to implement On-Demand Paging on the rxe driver, triple tasklets (requester, responder, and completer) must be allowed to sleep so that they can trigger page fault when pages being accessed are not present. This patch replaces the tasklets with workqueues, but still allows direct- call of

[RFC PATCH 1/7] IB/mlx5: Change ib_umem_odp_map_dma_single_page() to retain umem_mutex

2022-09-06 Thread Daisuke Matsuda
ib_umem_odp_map_dma_single_page(), which has been used only by the mlx5 driver, holds umem_mutex on success and releases on failure. This behavior is not convenient for other drivers to use it, so change it to always retain mutex on return. Signed-off-by: Daisuke Matsuda --- drivers/infiniband/c

[RFC PATCH 0/7] RDMA/rxe: On-Demand Paging on SoftRoCE

2022-09-06 Thread Daisuke Matsuda
Hi everyone, This patch series implements the On-Demand Paging feature on SoftRoCE(rxe) driver, which has been available only in mlx5 driver[1] so far. [Overview] When applications register a memory region(MR), RDMA drivers normally pin pages in the MR so that physical addresses are never changed

[RFC PATCH 4/7] RDMA/rxe: Add page invalidation support

2022-09-06 Thread Daisuke Matsuda
On page invalidation, an MMU notifier callback is invoked to unmap DMA addresses and update umem_odp->dma_list. The callback is registered when an ODP-enabled MR is created. Signed-off-by: Daisuke Matsuda --- drivers/infiniband/sw/rxe/Makefile | 3 ++- drivers/infiniband/sw/rxe/rxe_odp.c | 34