On Thu, Sep 7, 2017 at 2:13 PM, Declan Doherty <declan.dohe...@intel.com> wrote:
> On 07/09/2017 11:01 AM, Alejandro Lucero wrote: > >> I understand this is the representor idea suiting Intel cards but it does >> not cover other possibilities. >> >> At least, Netronome and Mellanox require a representor not just for >> controlling a VF, but also for sending and receiving packets through the >> representor PMD. >> >> > Hey Alejandro, > > What we've proposed here doesn't preclude the support of a data path on > the representor pmd as the functionality which the PMD supports is > dependent on how the parent device running the representor broker > configures each representor instance. As you note in the case of our > current generation of IO devices supporting a data path doesn't make sense, > but I think this framework could support that functionality if required. > > Hi Declan, Tell me if I'm wrong, but this looks to me like kernel netdev ndo_set_vf_* functions, maybe with the idea of having more possibilities than those currently available with the kernel. The representor idea I refer to is completely different, and it is related to using SRIOV with OVS offload. It has other possibilities but this is the current main reason. Some references: https://www.slideshare.net/Netronome/using-agilio-smartnics-for-openstack-networking-acceleration https://netdevconf.org/1.2/slides/oct6/04_gerlitz_efraim_introduction_to_switchdev_sriov_offloads.pdf I sent an abstract for a presentation in next Dublin Users meeting, and >> discussing the representor idea was in the agenda. >> >> > I've also submitted an presentation which I got notification was accepted > this morning which is based on the RFC, so maybe we should collaborate on a > presentation if there is a large overlap? Unfortunately, I did not get my abstract approved. Maybe you could add some slide about this other view for the shake of discussion. > > > By the way, I remember there was reticence about adding control plane to >> DPDK. What is the current official DPDK position in this regard? >> >> > I also remember there was concerns about adding control plane APIs to > DPDK, hence the RFC, but I think the advantage of the representor port > approach is that there are no new public API being introduced, with only > back-end support in the PMDs which wish to support this functionality. > > I'd like to know what is the opinion of the DPDK community regarding the control path. > > >> >> On Thu, Sep 7, 2017 at 9:35 AM, Mohammad Abdul Awal < >> mohammad.abdul.a...@intel.com> wrote: >> >> Signed-off-by: Mohammad Abdul Awal <mohammad.abdul.a...@intel.com> >>> Signed-off-by: Remy Horton <remy.hor...@intel.com> >>> Signed-off-by: Declan Doherty <declan.dohe...@intel.com> >>> --- >>> This RFC introduces a port representor based model for the control, >>> management >>> and monitoring of SR-IOV virtual function (VF) devices for control plane >>> applications which have bound the devices physical function. >>> >>> Port Representors are virtual poll mode drivers (PMD) which provide a >>> logical >>> representation in DPDK for VF ports for control and monitoring. Each port >>> representor PMD represents a single VF and is associated with it's parent >>> physical function (PF) PMD which provides the back-end hooks for the >>> representor >>> device ops and defines the control domain to which that port belongs.This >>> allows to use existing DPDK APIs to monitor and control the port without >>> the >>> need to create and maintain VF specific APIs. >>> >>> >>> +-----------------------------+ +---------------+ +---------------+ >>> | Control Plane | | Data Plane | | Data Plane | >>> | Application | | Application | | Application | >>> +-----------------------------+ +---------------+ +---------------+ >>> | eth dev api | | eth dev api | | eth dev api | >>> +-----------------------------+ +---------------+ +---------------+ >>> +-------+ +-------+ +-------+ +---------------+ +---------------+ >>> | PF0 | | Port | | Port | | VF0 PMD | | VF0 PMD | >>> | PMD <--+ Rep 0 | | Rep 1 | +---------------+ +------+--------+ >>> | | | PMD | | PMD | | >>> +---+--^+ +-------+ +-+-----+ | >>> | | | | | >>> | +----------------+ | | >>> | | | >>> | | | >>> +--------------------------------+ | >>> | | HW (logical view) | | | >>> | --+------+ +-------+ +---+---+ | | >>> | | PF | | VF0 | | VF1 | | | >>> | | | | | | +----------------------------+ >>> | +--------+ +-------+ +-------+ | >>> | +----------------------------+ | >>> | | VEB | | >>> | +----------------------------+ | >>> | +--------+ | >>> | | Port | | >>> | | 0 | | >>> | +--------+ | >>> +--------------------------------+ >>> >>> The figure above shows a deployment where the PF is bound to a DPDK >>> control >>> plane application which uses representor ports to manage the >>> configuration >>> and >>> monitoring of it's VF ports. Each virtual function is represented in the >>> application by a representor port PMD which enables control of the >>> corresponding >>> VF through eth dev APIs on the representor PMD such as: >>> >>> - void rte_eth_promiscuous_enable(uint8_t port_id); >>> - void rte_eth_promiscuous_disable(uint8_t port_id); >>> - void rte_eth_allmulticast_enable(uint8_t port_id); >>> - void rte_eth_allmulticast_disable(uint8_t port_id); >>> - int rte_eth_dev_mac_addr_add(uint8_t port, struct ether_addr >>> *mac_addr, >>> uint32_t pool); >>> - int rte_eth_dev_set_vlan_offload(uint8_t port_id, int offload_mask); >>> >>> as well as monitoring through API's like >>> >>> - void rte_eth_link_get(uint8_t port_id, struct rte_eth_link *link); >>> - int rte_eth_stats_get(uint8_t port_id, struct rte_eth_stats *stats); >>> >>> The port representor infrastructure is enabled through a single common, >>> device >>> independent, virtual PMD whos context is initialized and enabled through >>> a >>> broker instance running within the context of the physical function >>> device >>> driver. >>> >>> +-------------------------+ +-------------------------+ >>> | rte_ethdev | | rte_ethdev | >>> +-------------------------+ +-------------------------+ >>> | Physical Function PMD | | Port Reperesentor PMD | >>> | +-------------+ | | +---------+ +---------+ | >>> | | Representor | | | | dev_data| | dev_ops | | >>> | | Broker | | | +----+----+ +----+----+ | >>> | | +---------+ | | +------|-----------|------+ >>> | | | VF Port | | | | | >>> | | | Context +------------------+ | >>> | | +---------+ | | | >>> | | +---------+ | | | >>> | | | Handler +------------------------------+ >>> | | | Ops | | | >>> | | +---------+ | | >>> | +-------------+ | >>> +-------------------------+ >>> >>> Creation of representor ports can be achieved either through the --vdev >>> EAL >>> option or through the rte_vdev_init() API. Each port representor requires >>> the >>> BDF of it's parent PF and the Virtual Function ID of the port which the >>> representor will support. During initialization of the representor PMD, >>> it >>> calls >>> the broker API to register itself with the PF PMD and to get it's context >>> configured which includes the setting up of it's context and ops function >>> handlers. >>> >>> >>> As the port representor model is based around the paradigm of using >>> standard >>> port based APIs, it will allow future expansion of functionality without >>> the >>> need to add new APIs. For example it should be possible to support >>> configuration >>> of egress QoS parameters using existing TM APIs by extending the port >>> representor PMD/broker infrastructure. >>> >>> Mohammad Abdul Awal (5): >>> Implemented port representor broker infrastructure, created BDF to >>> port function. >>> added --enable-representor command line argument in EAL to load >>> representor broker infrastructure. >>> Implement port representor PMD >>> Enable port representor PMD and broker for fortville PMD driver >>> Enable port representor PMD and broker for ixgbe PMD driver. >>> >>> config/common_base | 5 + >>> drivers/net/Makefile | 2 + >>> drivers/net/i40e/Makefile | 1 + >>> drivers/net/i40e/i40e_ethdev.c | 17 + >>> drivers/net/i40e/i40e_prep_ops.c | 402 +++++++++ >>> drivers/net/i40e/i40e_prep_ops.h | 41 + >>> drivers/net/i40e/rte_pmd_i40e.c | 47 + >>> drivers/net/i40e/rte_pmd_i40e.h | 18 + >>> drivers/net/ixgbe/Makefile | 2 +- >>> drivers/net/ixgbe/ixgbe_ethdev.c | 33 +- >>> drivers/net/ixgbe/ixgbe_ethdev.h | 11 + >>> drivers/net/ixgbe/ixgbe_prep_ops.c | 279 ++++++ >>> drivers/net/ixgbe/ixgbe_prep_ops.h | 41 + >>> drivers/net/representor/Makefile | 51 ++ >>> drivers/net/representor/rte_eth_representor.c | 973 >>> +++++++++++++++++++++ >>> .../representor/rte_pmd_representor_version.map | 4 + >>> lib/librte_eal/bsdapp/eal/eal.c | 6 + >>> lib/librte_eal/common/eal_common_options.c | 1 + >>> lib/librte_eal/common/eal_internal_cfg.h | 2 + >>> lib/librte_eal/common/eal_options.h | 2 + >>> lib/librte_eal/common/include/rte_eal.h | 8 + >>> lib/librte_eal/linuxapp/eal/eal.c | 9 + >>> lib/librte_ether/Makefile | 2 + >>> lib/librte_ether/rte_ethdev.c | 93 ++ >>> lib/librte_ether/rte_ethdev.h | 26 + >>> lib/librte_ether/rte_ethdev_vdev.h | 37 +- >>> lib/librte_ether/rte_ether_version.map | 9 + >>> lib/librte_ether/rte_port_representor.c | 160 ++++ >>> lib/librte_ether/rte_port_representor.h | 289 ++++++ >>> mk/rte.app.mk | 1 + >>> 30 files changed, 2556 insertions(+), 16 deletions(-) >>> create mode 100644 drivers/net/i40e/i40e_prep_ops.c >>> create mode 100644 drivers/net/i40e/i40e_prep_ops.h >>> create mode 100644 drivers/net/ixgbe/ixgbe_prep_ops.c >>> create mode 100644 drivers/net/ixgbe/ixgbe_prep_ops.h >>> create mode 100644 drivers/net/representor/Makefile >>> create mode 100644 drivers/net/representor/rte_eth_representor.c >>> create mode 100644 drivers/net/representor/rte_ >>> pmd_representor_version.map >>> create mode 100644 lib/librte_ether/rte_port_representor.c >>> create mode 100644 lib/librte_ether/rte_port_representor.h >>> >>> -- >>> 2.7.4 >>> >>> >>> >> >