On Wed, Sep 1, 2021 at 11:55 AM Jerin Jacob <jerinjac...@gmail.com> wrote: > > On Wed, Sep 1, 2021 at 11:12 AM Kaladi, Ashok K > <ashok.k.kal...@intel.com> wrote: > > > > Dear dpdk-dev team, > > > > We would like to propose the following RFC for your review. > > > > A user space application may need access to the packets handled by eventdev > > based DPDK application. This application doesn't use mbuf or eventdev based > > DPDK APIs. Presently this is not possible without passing packets through > > DPDK KNI. > > > I think it is an innovative idea it is useful for multiple use cases > not just for eventdev. > > Some feedback on thoughts > > 1) The FIFO library should be generic it should not be specific to eventdev > 2) I think, This FIFO library should be generic and KNI also be a > consumer of this library > 3) I think, FIFO should not be a device instead it should be an > abstact object like rte_mempool * > 4) We need to consider User space app can be another DPDK primary > process or some non DPDK app > 4) I think, we can remove the Linux shared memory dependency instead > of introduce some > scheme of "exporting" memzone from DPDK application to another user > space app or another DPDK primary process. > I see the following reasons: > - It is backed by hugepage so better performance > - Is kernel do any memcpy when using Linux shm calls in kernel space? > > Thoughts? > > May be you can share set of API prototypes without any implementation > for the next level discussion if > others are OK this kind of library.
Some thoughts for generic fifo library. // Create SPSC fifo of size struct rte_ipc_queue* rte_ipc_queue_create(const char *name, size_t size, int socket_id, unsigned int flags); void rte_ipc_queue_destroy(struct rte_ipc_queue* iq); // userspace app or another DPDK primary process to get the ipc_queue handle. struct rte_ipc_queue* rte_ipc_queue_lookup(const char *proc_name, const char *ipc_queue_name); // Get the memory of size to fill by producer size_t rte_ipc_queue_enq_peek(struct rte_ipc_queue* iq, void **obj, size_t size); // Commit the head pointer int rte_ipc_queue_enq_commit(struct rte_ipc_queue* iq, void *obj, size_t size); // Get the memory of size to to consumer by consumer size_t rte_ipc_queue_deq_peek(struct rte_ipc_queue* iq, void **obj, size_t size); // Commit the tail pointer int rte_ipc_queue_deq_commit(struct rte_ipc_queue* iq, void *obj, size_t size); int rte_ipc_queue_in_use(struct rte_ipc_queue* iq) int rte_ipc_queue_avilable(struct rte_ipc_queue* iq) except rte_ipc_queue_create() and rte_ipc_queue_destroy() all the remaning rte_ipc_* APIs can be used the other userspace appliction. Another alternative is to have ZeroMQ kind of messaging library for various transports. It may not as fast this, Needs a prototype to verify. > > > > > This RFC introduces control packet event adapter (CP adapter) and FIFO > > library > > which can be used by user space applications to access the packets > > handled by eventdev based DPDK application. Linux shared memory is used to > > keep mempool (mbufs), packets and FIFOs. This allows copy-free access of > > packets to both the applications. The Rx and Tx FIFOs are used to exchange > > packets between the applications. > > > > '------------------' > > | mempool | > > '-------------' '-----------' |..................| '----------' > > '-----------' > > | | | | | <--- Tx FIFOs | | | > > | | > > | User space | | | | <--- Alloc FIFO | | Control | > > | DPDK | > > | App <----> FIFO Lib <----> ---> Rx FIFOs <----> Packet > > <----> Eventdev | > > | | | | | ---> Free FIFO | | Adapter | > > | App | > > | | | | | | | | > > | | > > '-------------' '-----------' | | '----------' > > '-----------' > > '------------------' > > > > CP adapter does the mbuf allocation and free for user space application. > > Mbufs > > allocated for user space application are put into alloc queue by CP adapter. > > The used mbufs are put into free queue by user space application. > > The FIFO lib translates packets between rte_mbuf format and a simplified > > format which will be used by the user application. FIFO lib doesn't use any > > other > > DPDK features than rte_mbuf structure. The simplified packet format > > contains only > > essential parameters like address of the payload, packet length, etc. > > > > - user space app send path: When user space application wants to send > > a packet, it takes an mbufs from Alloc FIFO, writes packet to it and puts > > it > > into Rx queue. CP adapter which polls this queue gets the packet and > > enqueues > > to eventdev. Then the mbuf is freed by CP adapter. > > > > - use space app receive path: CP adapter puts the packets to Tx FIFO, > > user space application polls the FIFO and gets the packet. After consuming > > the packet it puts the mbuf to be freed to free FIFO. CP adapter > > regularly > > reads the free FIFO and frees the mbufs in it. > > > > The CP adapter can service multiple devices and queues. Each queue is > > configured with a servicing weight to control the relative frequency with > > which > > the adapter polls the queue, and the event fields to use when constructing > > packet events. > > > > API Summary > > > > FIFO APIs: > > - FIFO device open/close. > > - API to allocate PDU buffer from alloc queue. > > - API to send a burst of packets to an Rx FIFO identified by FIFO id and > > device id. > > - API to receive a burst of packets from Tx FIFO identified by FIFO id and > > device id. > > - API to put the used buffer to Free queue. Use application will call this > > after > > using the payload. > > > > Control Packet Adapter APIs: > > - Initialization API. > > - Device addition and removal APIs > > - Device configuration APIs. > > - FIFO add and remove APIs. > > - Event enqueue and dequeue APIs. > > > > > > We look forward for your feedback on this proposal. Once we have initial > > feedback, > > patches will be submitted for review. > > > > > > Thanks & Regards > > Ashok Kaladi > >