Having taken feedback from the community into account, we would like to propose
some changes to our approach for combining multiple packet-processing functions
into a single operation on a single device, be that an optimized software
library or a hardware accelerator.
The main feedback on the rte_accelerator API can be summarized as follows:
1) Why are we creating another new library that performs tasks similar to other
existing APIs... why not try and converge on one?
2) The term "accelerator" is too broad a term, if the API is primarily focused
on Crypto + CRC
We also felt that using the rte_cryptodev and rte_compressdev APIs to
initialize, configure and reset the devices and then using rte_accelerator for
session creation and operation enqueue/dequeue was confusing matters.
We believe the new approach addresses the above concerns and also greatly
simplifies the solution.
Our new approach proposes to use the already existing rte_rawdev API with some
added functionality for creating "multi-function" sessions.
At the high level, the main changes are:
- The rte_accelerator library will no longer be added
- The rte_rawdev API will be used to initialize, configure, reset a
device
- A new rawdev interface for "multi-function" sessions/operations will
be added under the new directory 'drivers/raw/common'
- this interface's header file will be called
'rte_rawdev_multi_fn.h' (with an accompanying C file for function
implementations)
- this header file will contain much of what was previously
included in rte_accelerator.h and rte_err_detect.h, such as:
- enums and structs for defining a multi-function chain
of xforms, using xform definitions from rte_cryptodev and rte_compressdev as
necessary
- enums and structs for defining a multi-function chain
of ops, again using op definitions from rte_cryptodev and rte_compressdev as
necessary
- enums and structs for defining error-detection xforms
and ops
- two API function definitions to create and destroy a
session based on a xform chain
- rte_rawdev_multi_fn_session_create()
- rte_rawdev_multi_fn_session_destroy()
- application code will include rte_rawdev_multi_fn.h to access
the structs, enums and functions for creating xform chains, sessions and op
chains
- keeping the multi-function interface under the 'drivers'
directory means that rte_rawdev itself remains completely "raw", with no
knowledge of xforms, sessions or ops
- a proposal for this header file is included at the end
- The rte_rawdev API will be used to enqueue/dequeue the operations
using the existing rte_rawdev_enqueue_buffers() and rte_rawdev_dequeue_buffers()
- a synchronous API function could potentially be added to
rte_rawdev in the future if required, to avoid the overhead of enqueue/dequeue
for the optimized software library use-case (e.g. rte_rawdev_process_buffers())
- Two new rawdev PMDs for will be added under 'drivers/raw' for QAT and
AESNI-MB
- these two rawdev PMDs will use and implement the
multi-function interface defined in 'drivers/raw/common/rte_rawdev_multi_fn.h'
- as with all other rawdev PMDs, the interface is known only to
the application and the PMD itself, and is opaque to rte_rawdev itself
- the PMDs will be added under 'drivers/raw/aesni_mb' and
'drivers/raw/qat'
- other PMDs (existing or new) could use this multi-function
interface in the future if use-cases arise
- The rte_rawdev library will be used as is, with no changes required
The initial use cases for the multi-function rawdev interface remain the same
as for the previously proposed rte_accelerator:
- DOCSIS MAC: Crypto + CRC
- XGS-PON MAC: Crypto + CRC + BIP
However, the API can still also accommodate other chained functions such as
Compression + Crypto and UDP Checksum + Crypto.
The following diagram shows the new architecture:
+-----------------------------------------------------------+
| |
| Application |
| (e.g. vCMTS (DOCSIS), vOLT (XGS-PON), etc.) |
| |
+-----------------------------------------------------------+
|
+---------------------------|-------------------------------+
| | DPDK |
| | |
| +---------------------+ |
| | | |
| | rte_rawdev | |
| | | |
NOTE:
| +---------------------+ ____________|_______
'RAWDEV MULTI-FUNCTION
| / \ / |
API' is opaque to
| / \ / |
rte_rawdev
| / \ / |
| +--------------------------------+ |
| | RAWDEV MULTI-FUNCTION API | |
| +--------------------------------+ |
| +------------+ +------------+ |
| | RAWDEV | | RAWDEV | |
| | AESNI-MB | | QAT | |
| | PMD | | PMD | |
| +------------+ +------------+ |
| | | |
+------------------|------------------|---------------------+
| |
+------------+ +------------+
| AESNI-MB | | QAT HW |
| SW LIB | | |
+------------+ +------------+
Note that development work is already progressing well on this new approach,
with the aim of upstreaming this into DPDK v20.05.
Also, we may consider consolidating the multi-function API into the main DPDK
library in the future if there were more devices which wanted to support these
multi-function operations.
The following is the proposed rawdev multi-function interface, defined in
'drivers/raw/common/rte_rawdev_multi_fn.h'
/* SPDX-License-Identifier: BSD-3-Clause
* Copyright(c) 2020 Intel Corporation.
*/
#ifndef _RTE_RAWDEV_MULTI_FN_H_
#define _RTE_RAWDEV_MULTI_FN_H_
#ifdef __cplusplus
extern "C" {
#endif
#include <rte_compat.h>
#include <rte_common.h>
#include <rte_mbuf.h>
#include <rte_memory.h>
#include <rte_mempool.h>
#include <rte_comp.h>
#include <rte_crypto.h>
#include <rte_rawdev.h>
/** Error Detection Algorithms */
enum rte_rawdev_multi_fn_err_detect_algorithm {
RTE_RAWDEV_MULTI_FN_ERR_DETECT_CRC32_ETH,
/**< CRC32 Ethernet */
RTE_RAWDEV_MULTI_FN_ERR_DETECT_BIP32
/**< BIP32 */
};
/** Error Detection Operation Types */
enum rte_rawdev_multi_fn_err_detect_operation {
RTE_RAWDEV_MULTI_FN_ERR_DETECT_OP_VERIFY,
/**< Verify error detection result */
RTE_RAWDEV_MULTI_FN_ERR_DETECT_OP_GENERATE
/**< Generate error detection result */
};
/** Error Detection Status */
enum rte_rawdev_multi_fn_err_detect_op_status {
RTE_RAWDEV_MULTI_FN_ERR_DETECT_OP_STATUS_NOT_PROCESSED,
/**< Operation has not yet been processed by a device */
RTE_RAWDEV_MULTI_FN_ERR_DETECT_OP_STATUS_SUCCESS,
/**< Operation completed successfully */
RTE_RAWDEV_MULTI_FN_ERR_DETECT_OP_STATUS_VERIFY_FAILED,
/**< Verification failed */
RTE_RAWDEV_MULTI_FN_ERR_DETECT_OP_STATUS_ERROR
/**< Error handling operation */
};
/**
* Error Detection Transform Data
*
* This structure contains data relating to an error detection transform. The
* fields *op* and *algo* are common to all error detection transforms and
* MUST be set
*/
struct rte_rawdev_multi_fn_err_detect_xform {
enum rte_rawdev_multi_fn_err_detect_operation op;
/**< Error detection operation type */
enum rte_rawdev_multi_fn_err_detect_algorithm algo;
/**< Error detection algorithm */
};
/** Error Detection Operation */
struct rte_rawdev_multi_fn_err_detect_op {
struct rte_mbuf *m_src; /**< Source mbuf */
enum rte_rawdev_multi_fn_err_detect_op_status status;
/**< Operation status */
struct {
uint16_t offset;
/**<
* Starting point for error detection processing, specified
* as the number of bytes from start of the packet in the
* source mbuf
*/
uint16_t length;
/**<
* The length, in bytes, of the source mbuf on which the error
* detection operation will be computed
*/
} data; /**< Data offset and length for error detection */
struct {
uint8_t *data;
/**<
* This points to the location where the error detection
* result should be written (in the case of generation) or
* where the purported result exists (in the case of
* verification)
*
* The caller must ensure the required length of physically
* contiguous memory is available at this address
*
* For a CRC, this may point into the mbuf packet data. For
* an operation such as a BIP, this may point to a memory
* location after the op
*
* For generation, the result will overwrite any data at this
* location
*/
rte_iova_t phys_addr;
/**< Physical address of output data */
} output; /**< Output location */
};
/**
* Multi-function transform types
*/
enum rte_rawdev_multi_fn_xform_type {
RTE_RAWDEV_MULTI_FN_XFORM_TYPE_CRYPTO_SYM,
/**< Symmetric crypto transform type */
RTE_RAWDEV_MULTI_FN_XFORM_TYPE_CRYPTO_ASYM,
/**< Asymmetric crypto transform type */
RTE_RAWDEV_MULTI_FN_XFORM_TYPE_COMP,
/**< Compression transform type */
RTE_RAWDEV_MULTI_FN_XFORM_TYPE_ERR_DETECT
/**< Error detection transform type */
};
/**
* Multi-function transform setup data
*
* This structure is used to specify the multi-function transforms required.
* Multiple transforms can be chained together to specify a chain of transforms
* such as symmetric crypto followed by error detection, or compression followed
* by symmetric crypto. Each transform structure holds a single transform, with
* the type field specifying which transform is contained within the union.
*/
struct rte_rawdev_multi_fn_xform {
struct rte_rawdev_multi_fn_xform *next;
/**<
* Next transform in the chain
* - the last transform in the chain MUST set this to NULL
*/
enum rte_rawdev_multi_fn_xform_type type;
/**< Transform type */
RTE_STD_C11
union {
struct rte_crypto_sym_xform crypto_sym;
/**< Symmetric crypto transform */
struct rte_crypto_asym_xform crypto_asym;
/**< Asymmetric crypto transform */
struct rte_comp_xform comp;
/**< Compression transform */
struct rte_rawdev_multi_fn_err_detect_xform err_detect;
/**< Error detection transform */
};
};
/**
* Multi-function operation status
*/
enum rte_rawdev_multi_fn_op_status {
RTE_RAWDEV_MULTI_FN_OP_STATUS_SUCCESS = 0,
/**< Operation completed successfully */
RTE_RAWDEV_MULTI_FN_OP_STATUS_FAILURE,
/**< Operation completed with failure */
RTE_RAWDEV_MULTI_FN_STATUS_INVALID_SESSION,
/**< Operation failed due to invalid session arguments */
RTE_RAWDEV_MULTI_FN_OP_STATUS_NOT_PROCESSED,
/**< Operation has not yet been processed by a device */
};
/**
* Multi-function session data
*/
struct rte_rawdev_multi_fn_session;
/**
* Multi-function operation data
*
* This structure is used to specify the operations for a particular session.
* This includes specifying the source and, if required, destination mbufs and
* the lengths and offsets of the data within these mbufs on which the
* operations should be done. Multiple operations are chained together to
* specify the full set of operations to be performed
*
* @note The rte_rawdev_multi_fn_op chain MUST match the session's xform
* chain exactly
* @note The first rte_rawdev_multi_fn_op element in the chain is the parent
* operation. The following fields MUST be set in this first operation before
* enqueuing and are ignored in the inner operations and any subsequent
* rte_rawdev_multi_fn_op chain elements:
* - *sess*
* - *m_src*
* - *m_dst* (if required)
* @note If *sess* or *m_src* is not set in the first rte_rawdev_multi_fn_op,
* this operation is invalid and will cause an error when attempting to enqueue.
* @note The following fields MUST be set in ALL rte_rawdev_multi_fn_op chain
* elements:
* - *next*
* - *mempool*
* - *type*
* @note After the operation has been dequeued, only the FIRST (i.e. the parent)
* rte_rawdev_multi_fn_op in the chain will contain the *overall_status*. Each
* chain element will contain it's individual *op_status*, the value of which is
* relevant to operation type (e.g. an ::rte_crypto_op_status,
* ::rte_comp_op_status or ::rte_err_detect_op_status)
*/
struct rte_rawdev_multi_fn_op {
struct rte_rawdev_multi_fn_op *next;
/**<
* Next operation in the chain
* - the last operation in the chain MUST set this to NULL
*/
struct rte_rawdev_multi_fn_session *sess;
/**< Handle for the associated multi-function session */
struct rte_mempool *mempool;
/**< Mempool from which the operation is allocated */
struct rte_mbuf *m_src; /**< Source mbuf */
struct rte_mbuf *m_dst; /**< Destination mbuf */
enum rte_rawdev_multi_fn overall_status;
/**<
* Overall operation status
* - indicates if all the operations in the chain succeeded or if any
* one of them failed
*/
uint8_t op_status;
/**<
* Individual operation status
* - indicates the status of the individual operation in the chain
*/
RTE_STD_C11
union {
struct rte_crypto_sym_op crypto_sym;
/**< Symmetric crypto operation */
struct rte_crypto_asym_op crypto_asym;
/**< Asymmetric crypto operation */
struct rte_comp_op comp;
/**< Compression operation */
struct rte_rawdev_multi_fn_err_detect_op err_detect;
/**< Error detection operation */
};
};
/**
* Create multi-function session as specified by the transform chain
*
* @param dev_info Device info, obtained by calling rte_rawdev_info_get()
* @param xform Pointer to the first element of the session transform
* chain
* @param socket_id Socket to allocate the session on
*
* @return
* - Pointer to session, if successful
* - NULL, on failure
*/
__rte_experimental
struct rte_rawdev_multi_fn_session *
rte_rawdev_multi_fn_session_create(struct rte_rawdev_info *dev_info,
struct rte_rawdev_multi_fn_xform *xform,
int socket_id);
/**
* Free memory assoicated with a multi-function session
*
* @param dev_info Device info, obtained by calling rte_rawdev_info_get()
* @param sess Multi-function session to be freed
*
* @return
* - 0, if successful
* - -EINVAL, if session is NULL
* - -EBUSY, if not all session data has been freed
*/
__rte_experimental
int
rte_rawdev_multi_fn_session_destroy(struct rte_rawdev_info *dev_info,
struct rte_rawdev_multi_fn_session *sess);
#ifdef __cplusplus
}
#endif
#endif /* _RTE_RAWDEV_MULTI_FN_H_ */
