On 2/16/2015 12:15 PM, Tetsuya Mukawa wrote:
> This patch series adds a dynamic port hotplug framework to DPDK.
> With the patches, DPDK apps can attach or detach ports at runtime.
>
> The basic concept of the port hotplug is like followings.
> - DPDK apps must have responsibility to manage ports.
> DPDK apps only know which ports are attached or detached at the moment.
> The port hotplug framework is implemented to allow DPDK apps to manage
> ports.
> For example, when DPDK apps call port attach function, attached port number
> will be returned. Also, DPDK apps can detach port by port number.
> - Kernel support is needed for attaching or detaching physical device ports.
> To attach a new physical device port, the device will be recognized by
> userspace directly I/O framework in kernel at first. Then DPDK apps can
> call the port hotplug functions to attach ports.
> For detaching, steps are vice versa.
> - Before detach ports, ports must be stopped and closed.
> DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() before
> detaching ports. These function will call finalization codes of PMDs.
> But so far, no PMD frees all resources allocated by initialization.
> It means PMDs are needed to be fixed to support the port hotplug.
> 'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching.
> Without this flag, detaching will be failed.
> - Mustn't affect legacy DPDK apps.
> No DPDK EAL behavior is changed, if the port hotplug functions are't called.
> So all legacy DPDK apps can still work without modifications.
>
> And a few limitations.
> - The port hotplug functions are not thread safe.
> DPDK apps should handle it.
> - Only support Linux and igb_uio so far.
> BSD and VFIO is not supported. I will send VFIO patches at least, but I
> don't
> have a plan to submit BSD patch so far.
>
>
> Here is port hotplug APIs.
> -------------------------------------------------------------------------------
> /**
> * Attach a new device.
> *
> * @param devargs
> * A pointer to a strings array describing the new device
> * to be attached. The strings should be a pci address like
> * '0000:01:00.0' or virtual device name like 'eth_pcap0'.
> * @param port_id
> * A pointer to a port identifier actually attached.
> * @return
> * 0 on success and port_id is filled, negative on error
> */
> int rte_eal_dev_attach(const char *devargs, uint8_t *port_id);
>
> /**
> * Detach a device.
> *
> * @param port_id
> * The port identifier of the device to detach.
> * @param addr
> * A pointer to a device name actually detached.
> * @return
> * 0 on success and devname is filled, negative on error
> */
> int rte_eal_dev_detach(uint8_t port_id, char *devname);
> -------------------------------------------------------------------------------
>
> This patch series are for DPDK EAL. To use port hotplug function by DPDK apps,
> each PMD should be fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Please
> check
> a patch for pcap PMD.
>
> Also, please check testpmd patch. It will show you how to fix your legacy
> applications to support port hotplug feature.
>
> PATCH v8 changes
> - Fix Makefile and add version map file.
> - Add missing symbol in version map.
> - Fix pci_scan_one() to update sysfs values.
> (Thanks to Qiu, Michael and Iremonger, Bernard)
> - NONE_TRACE is replaced by NO_TRACE.
> - Fix typo.
> - Add size parameter to rte_eth_dev_save().
> (Thanks to Iremonger, Bernard)
>
> PATCH v7 changes
> - Add a new section to programmer's guide.
> (Thanks to Iremonger, Bernard)
> - Fix port checking implementation of star_port().
> - Fix typo of warning messages.
> - Add pt_driver checking to rte_eth_dev_check_detachable().
> (Thanks to Qiu, Michael)
>
> PATCH v6 changes
> - Fix rte_eth_dev_uninit() to handle a return value of uninit
> function of PMD. To do this, below changes also be applied.
> - Fix a parameter of rte_eth_dev_free().
> - Use rte_eth_dev structure as the paramter of rte_eth_dev_free().
>
> PATCH v5 changes
> - Add runtime check passthrough driver type, like vfio-pci, igb_uio
> and uio_pci_generic.
> This was done by Qiu, Michael. Thanks a lot.
> - Change function names like below.
> - rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke().
> - rte_eal_dev_invoke() to rte_eal_vdev_invoke().
> - Add code to handle a return value of rte_eal_devargs_remove().
> - Fix pci address format in rte_eal_dev_detach().
> - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused.
> - Change function definition of rte_eal_devargs_remove().
> - Fix pci_unmap_device() to check pt_driver.
> - Fix return value of below functions.
> - rte_eth_dev_get_changed_port().
> - rte_eth_dev_get_port_by_addr().
> - Change paramters of rte_eth_dev_validate_port() to cleanup code.
> - Fix pci_scan_one to handle pt_driver correctly.
> (Thanks to Qiu, Michael for above suggestions)
>
> PATCH v4 changes
> - Merge patches to review easier.
> - Fix indent of 'if' statement.
> - Fix calculation method of eal_compare_pci_addr().
> - Fix header file declaration.
> - Add header file to determine if hotplug can be enabled.
> (Thanks to Qiu, Michael)
> - Use braces with 'for' loop.
> - Add parameter checking.
> - Fix sanity check code
> - Fix comments of rte_eth_dev_type.
> - Change function names.
> (Thanks to Iremonger, Bernard)
>
> PATCH v3 changes:
> - Fix enum definition used in rte_ethdev.c.
> (Thanks to Zhang, Helin)
>
> PATCH v2 changes:
> - Replace rte_eal_dev_attach_pdev(), rte_eal_dev_detach_pdev,
> rte_eal_dev_attach_vdev() and rte_eal_dev_detach_vdev() to
> rte_eal_dev_attach() and rte_eal_dev_detach().
> - Add parameter values checking.
> - Refashion a few functions.
> (Thanks to Iremonger, Bernard)
>
> PATCH v1 Changes:
> - Fix error checking code of librte_eth APIs.
> - Fix issue that port from pcap PMD cannot be detached correctly.
> - Fix issue that testpmd could hang after forwarding, if attaching and
> detaching
> is repeatedly.
> - Fix if-condition of rte_eth_dev_get_port_by_addr().
> (Thanks to Mark Enright)
>
> RFC PATCH v2 Changes:
> - remove 'rte_eth_dev_validate_port()', and cleanup codes.
>
>
> Michael Qiu (2):
> eal_pci: Add flag to hold kernel driver type
> eal_pci: pci memory map work with driver type
>
> Tetsuya Mukawa (12):
> eal/pci,ethdev: Remove assumption that port will not be detached
> eal/pci: Consolidate pci address comparison APIs
> ethdev: Add rte_eth_dev_free to free specified device
> eal,ethdev: Add a function and function pointers to close ether device
> ethdev: Add functions that will be used by port hotplug functions
> eal/linux/pci: Add functions for unmapping igb_uio resources
> eal/pci: Add a function to remove the entry of devargs list
> eal/pci: Cleanup pci driver initialization code
> ethdev: Add one dev_type parameter to rte_eth_dev_allocate
> eal/pci: Add rte_eal_dev_attach/detach() functions
> eal: Enable port hotplug framework in Linux
> doc: Add port hotplug framework section to programmers guide
>
> app/test/virtual_pmd.c | 2 +-
> config/common_linuxapp | 5 +
> doc/guides/prog_guide/index.rst | 1 +
> doc/guides/prog_guide/port_hotplug_framework.rst | 110 ++++
> lib/librte_eal/bsdapp/eal/eal_pci.c | 29 +-
> lib/librte_eal/common/Makefile | 1 +
> lib/librte_eal/common/eal_common_dev.c | 276 ++++++++++
> lib/librte_eal/common/eal_common_devargs.c | 60 +++
> lib/librte_eal/common/eal_common_pci.c | 92 +++-
> lib/librte_eal/common/eal_private.h | 35 ++
> lib/librte_eal/common/include/rte_dev.h | 33 ++
> lib/librte_eal/common/include/rte_dev_hotplug.h | 44 ++
> lib/librte_eal/common/include/rte_devargs.h | 21 +
> lib/librte_eal/common/include/rte_pci.h | 84 +++
> lib/librte_eal/linuxapp/eal/Makefile | 1 +
> lib/librte_eal/linuxapp/eal/eal_pci.c | 232 +++++++--
> lib/librte_eal/linuxapp/eal/eal_pci_init.h | 8 +
> lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 67 ++-
> lib/librte_eal/linuxapp/eal/rte_eal_version.map | 2 +
> lib/librte_ether/rte_ethdev.c | 636
> +++++++++++++----------
> lib/librte_ether/rte_ethdev.h | 151 +++++-
> lib/librte_ether/rte_ether_version.map | 7 +
> lib/librte_pmd_af_packet/rte_eth_af_packet.c | 2 +-
> lib/librte_pmd_bond/rte_eth_bond_api.c | 2 +-
> lib/librte_pmd_pcap/rte_eth_pcap.c | 2 +-
> lib/librte_pmd_ring/rte_eth_ring.c | 2 +-
> lib/librte_pmd_xenvirt/rte_eth_xenvirt.c | 2 +-
> 27 files changed, 1560 insertions(+), 347 deletions(-)
> create mode 100644 doc/guides/prog_guide/port_hotplug_framework.rst
> create mode 100644 lib/librte_eal/common/include/rte_dev_hotplug.h
>
Acked-by: Michael Qiu <michael.qiu at intel.com>
