> -----Original Message-----
> From: Vamsi Krishna Attunuru <[email protected]>
> Sent: Wednesday, September 24, 2025 9:45 AM
> To: fengchengwen <[email protected]>; [email protected];
> [email protected]
> Cc: [email protected]; [email protected];
> [email protected]; [email protected];
> [email protected]; Jerin Jacob <[email protected]>
> Subject: RE: [EXTERNAL] Re: [RFC] lib/dma: introduce inter-process and
> inter-OS
> DMA
>
> Hi Feng, Anatoly,
>
> Gentle ping for below review.
>
> >-----Original Message-----
> >From: Vamsi Krishna Attunuru
> >Sent: Monday, September 22, 2025 5:19 PM
> >To: fengchengwen <[email protected]>; [email protected];
> >[email protected]
> >Cc: [email protected]; [email protected];
> >[email protected]; [email protected];
> >[email protected]; Jerin Jacob <[email protected]>
> >Subject: RE: [EXTERNAL] Re: [RFC] lib/dma: introduce inter-process and
> >inter- OS DMA
> >
> >Hi Feng,
> >
> >>Hi Vamsi, This commit change is more than discussed, it add control
> >>API which for group management. 1. Control API: I check this commit
> >>and Intel commit [1], it seem has a quite difference. I hope Intel
> >>guys can express views
> >>ZjQcmQRYFpfptBannerEnd
> >>Hi Vamsi,
> >>
> >>This commit change is more than discussed, it add control API which
> >>for group management.
> >>
> >>1. Control API: I check this commit and Intel commit [1], it seem has
> >>a quite difference.
> >> I hope Intel guys can express views. I prefer not add this part if
> >>no
> >response.
> >
> >This new feature needs to be securely managed through control APIs. It
> >would be extremely helpful if the folks at Intel and you as well could
> >provide support or inputs on this.
Beyond adding Intel folks to this thread, I don't see any further steps we can
take to drive review at this stage.
That said, the table-based concept used in the current API may not be portable,
and we may need improvements here.
Based on my understanding, DMA devices used for inter process copy can be
classified into three categories:
Class A: Requires a pair of DMA devices (one on each end of process/domain) for
data transfer. Marvell DMA devices fall into this category.
Class B: Requires only a single DMA device (one process/domain has a DMA
device, the other process does not). Intel DMA devices fall here.
Class C: Other types of devices that we are not yet aware of.
Abstracting all of these under a single API will be challenging. Linux and
other OSes do not provide control-plane APIs for this,
so DPDK must provide control plane mechanisms to support Class A, Class B, and
Class C devices.
Proposal: Split development into separate sets:
-----------------------------------------------
Set A: Focus only on the datapath. Assume uint16_t *src_handle and uint16_t
*dst_handle come from elsewhere (Class C).
Set B: Introduce capabilities for Class A devices with portable APIs (proposal
below, without table concept).
Set C: Introduce capabilities for Class B devices and relevant APIs, to be
added when needed.
We can merge Set A in the current release and move Set B to a next release _if_
review or support for Class A devices requires more time.
@fengchengwen Thoughts?
Class A API Proposal:
---------------------
These APIs are based on a new capability flag for inter-process or inter-OS DMA
transfers for Class A devices.
/** Creates an access group for pair-type inter-process or inter-OS DMA
transfers. */
int rte_dma_access_pair_group_create(const struct rte_dma_dev *dev,
rte_uuid_t process_id,
rte_uuid_t token,
uint16_t *group_id);
/** Destroys an access group once all participating devices have exited. */
int rte_dma_access_pair_group_destroy(const struct rte_dma_dev *dev,
uint16_t group_id);
/** Allows a device to join an existing access group using a device handle and
token. */
int rte_dma_access_pair_group_join(const struct rte_dma_dev *dev,
uint16_t group_id,
rte_uuid_t process_id,
rte_uuid_t token,
rte_dma_access_pair_leave_cb_t leave_cb);
/** Removes a device from an access group. */
int rte_dma_access_pair_group_leave(const struct rte_dma_dev *dev,
uint16_t group_id);
/** Retrieves the source and destination handles for a given device within the
group. */
int rte_dma_access_pair_gorup_src_dst_handles_get(const struct rte_dma_dev *dev,
uint16_t group_id,
rte_uuid_t src_process_id,
rte_uuid_t dst_process_id,
uint16_t *src_handle,
uint16_t *dst_handle);
Parameters that need explanation:
--------------------------------
process_id: Unique ID for the process, generated via rte_uuid_ APIs
token: Provided by an administrative actor to grant access, similar to VFIO VF
token creation used in VFIO PF driver.
leave_cb: Callback to notify when another side leaves the group
Example Workflow for Class A Inter-Domain DMA Transfer:
-------------------------------------------------------
This example demonstrates how three processes — p0, p1, and p2 — coordinate
inter-domain DMA transfers using pair-type(Class A) DMA devices.
Step 1: Group Creation (p0)
Process p0 calls rte_dma_access_pair_group_create() with a unique process
handle and token. A group_id is returned.
Step 2: Group Sharing
group_id and token are shared with p1 and p2 via IPC or shared memory.
Step 3: Group Joining (p1 & p2)
Processes p1 and p2 call rte_dma_access_pair_group_join() with their process id
and the shared token from admin
Step 4: Handle Discovery
Each process uses rte_dma_access_pair_gorup_src_dst_handles_get() to retrieve
source and destination handles for other processes.
Step 5: Transfer Coordination
Using the handles, each process configures a virtual channel (vchan) and
initiates DMA transfers.
Step 6: Group Teardown
When a process no longer needs to participate, it calls
rte_dma_access_pair_group_leave(). Other processes are notified via the
registered callback with rte_dma_access_pair_group_join().
Once all devices have exited, p0 calls rte_dma_access_pair_group_destroy() to
clean up.
For Class B: We can add new capability flag and have new set of APIs
rte_dma_access_master_ or so. When such devices comes or when intel wants to
add it
