Hi Feng, Bruce, Kevin, Could you please review the proposal below and share your thoughts. we are seeking your feedback on this feature to ensure alignment before the rc deadlines.
Regards Vamsi >>> -----Original Message----- >>> From: Vamsi Krishna Attunuru <[email protected]> >>> >From: Vamsi Krishna Attunuru >>> >Sent: Monday, September 22, 2025 5:19 PM >>> >To: fengchengwen <[email protected]>; [email protected]; >>> >[email protected] >>> >Cc: [email protected]; [email protected]; >>> >[email protected]; [email protected]; >>> >[email protected]; Jerin Jacob <[email protected]> >>> >Subject: RE: [EXTERNAL] Re: [RFC] lib/dma: introduce inter-process >>> >and >>> >inter- OS DMA >>> > >>> >Hi Feng, >>> > >>> >>Hi Vamsi, This commit change is more than discussed, it add control >>> >>API which for group management. 1. Control API: I check this commit >>> >>and Intel commit [1], it seem has a quite difference. I hope Intel >>> >>guys can express views ZjQcmQRYFpfptBannerEnd Hi Vamsi, >>> >> >>> >>This commit change is more than discussed, it add control API which >>> >>for group management. >>> >> >>> >>1. Control API: I check this commit and Intel commit [1], it seem >>> >>has a quite difference. >>> >> I hope Intel guys can express views. I prefer not add this part >>> >>if no >>> >response. >>> > >>> >This new feature needs to be securely managed through control APIs. >>> >It would be extremely helpful if the folks at Intel and you as well >>> >could provide support or inputs on this. >> >> >>Beyond adding Intel folks to this thread, I don't see any further steps >>we can take to drive review at this stage. >> >>That said, the table-based concept used in the current API may not be >>portable, and we may need improvements here. >> >>Based on my understanding, DMA devices used for inter process copy can >>be classified into three categories: >> >>Class A: Requires a pair of DMA devices (one on each end of >>process/domain) for data transfer. Marvell DMA devices fall into this >category. >>Class B: Requires only a single DMA device (one process/domain has a >>DMA device, the other process does not). Intel DMA devices fall here. >>Class C: Other types of devices that we are not yet aware of. >> >>Abstracting all of these under a single API will be challenging. Linux >>and other OSes do not provide control-plane APIs for this, so DPDK must >>provide control plane mechanisms to support Class A, Class B, and Class C >devices. >> >>Proposal: Split development into separate sets: >>----------------------------------------------- >>Set A: Focus only on the datapath. Assume uint16_t *src_handle and >>uint16_t *dst_handle come from elsewhere (Class C). >>Set B: Introduce capabilities for Class A devices with portable APIs >>(proposal below, without table concept). >>Set C: Introduce capabilities for Class B devices and relevant APIs, to >>be added when needed. >> >>We can merge Set A in the current release and move Set B to a next >>release _if_ review or support for Class A devices requires more time. >> >>@fengchengwen Thoughts? >> >>Class A API Proposal: >>--------------------- >>These APIs are based on a new capability flag for inter-process or >>inter-OS DMA transfers for Class A devices. >> >> >>/** Creates an access group for pair-type inter-process or inter-OS DMA >>transfers. */ int rte_dma_access_pair_group_create(const struct >>rte_dma_dev *dev, >> rte_uuid_t process_id, >> rte_uuid_t token, >> uint16_t *group_id); >> >>/** Destroys an access group once all participating devices have >>exited. */ int rte_dma_access_pair_group_destroy(const struct >rte_dma_dev *dev, >> uint16_t group_id); >> >>/** Allows a device to join an existing access group using a device >>handle and token. */ int rte_dma_access_pair_group_join(const struct >>rte_dma_dev *dev, >> uint16_t group_id, >> rte_uuid_t process_id, >> rte_uuid_t token, >> rte_dma_access_pair_leave_cb_t >>leave_cb); >> >>/** Removes a device from an access group. */ int >>rte_dma_access_pair_group_leave(const struct rte_dma_dev *dev, >> uint16_t group_id); >> >>/** Retrieves the source and destination handles for a given device >>within the group. */ int >>rte_dma_access_pair_gorup_src_dst_handles_get(const struct >rte_dma_dev *dev, >> uint16_t group_id, >> rte_uuid_t src_process_id, >> rte_uuid_t dst_process_id, >> uint16_t *src_handle, >> uint16_t *dst_handle); >> >> >>Parameters that need explanation: >>-------------------------------- >>process_id: Unique ID for the process, generated via rte_uuid_ APIs >>token: Provided by an administrative actor to grant access, similar to >>VFIO VF token creation used in VFIO PF driver. >>leave_cb: Callback to notify when another side leaves the group >> >> >>Example Workflow for Class A Inter-Domain DMA Transfer: >>------------------------------------------------------- >> >>This example demonstrates how three processes — p0, p1, and p2 — >>coordinate inter-domain DMA transfers using pair-type(Class A) DMA >devices. >> >>Step 1: Group Creation (p0) >>Process p0 calls rte_dma_access_pair_group_create() with a unique >>process handle and token. A group_id is returned. >> >>Step 2: Group Sharing >>group_id and token are shared with p1 and p2 via IPC or shared memory. >> >>Step 3: Group Joining (p1 & p2) >>Processes p1 and p2 call rte_dma_access_pair_group_join() with their >>process id and the shared token from admin >> >>Step 4: Handle Discovery >>Each process uses rte_dma_access_pair_gorup_src_dst_handles_get() to >>retrieve source and destination handles for other processes. >> >>Step 5: Transfer Coordination >>Using the handles, each process configures a virtual channel (vchan) >>and initiates DMA transfers. >> >>Step 6: Group Teardown >>When a process no longer needs to participate, it calls >>rte_dma_access_pair_group_leave(). Other processes are notified via the >>registered callback with rte_dma_access_pair_group_join(). >>Once all devices have exited, p0 calls >>rte_dma_access_pair_group_destroy() >>to clean up. >> >> >>For Class B: We can add new capability flag and have new set of APIs >>rte_dma_access_master_ or so. When such devices comes or when intel >>wants to add it >> >> >> >>
