Hi, I just sent v6: http://openvswitch.org/pipermail/dev/2016-May/071786.html
On Thu, May 19, 2016 at 9:30 AM, Mauricio Vásquez < mauricio.vasquezber...@studenti.polito.it> wrote: > > > On Tue, May 17, 2016 at 12:44 AM, Flavio Leitner <f...@sysclose.org> wrote: > >> On Fri, May 13, 2016 at 10:25:41PM +0200, Mauricio Vasquez B wrote: >> > In order to use dpdk ports in ovs they have to be bound to a DPDK >> > compatible driver before ovs is started. >> > >> > This patch adds the possibility to hotplug (or hot-unplug) a device >> > after ovs has been started. The implementation adds two appctl commands: >> > netdev-dpdk/port-attach and netdev-dpdk/port-detach >> > >> > After the user attaches a new device, it has to be added to a bridge >> > using the add-port command, similarly, before detaching a device, >> > it has to be removed using the del-port command. >> > >> > Signed-off-by: Mauricio Vasquez B < >> mauricio.vasquezber...@studenti.polito.it> >> >> The patch looks good, but it didn't work for me with VFIO (it's >> not supported by the DPDK hotplug framework). Since the doc >> includes instructions to use VFIO, should we add a note for the >> lack of hotplug support? >> >> > Yes, we should. I'll wait few days to see if there is more feedback and > I'll send a new version including that note. > > >> The attach/detach operations seem to work with uio_pci_generic, >> but it doesn't move packets. The interface statistics shows >> only 'errs' increasing a lot and few 'drops'. >> >> Kernel: 4.5.0+ >> OVS: master (b396293a) + patch >> DPDK: 16.04 >> Drivers: vfio-pci or uio_pci_generic >> Hardware: 82599ES >> >> > I tried uio_pci_generic in the following conditions without any problem. > > Kernel: 3.19.0-32-generic > OVS: master(d640a7e1) + patch > DPDK: 16.04 > Hardware: X540-AT2 > > >> Do you know if there are patches/plans to support that with VFIO or >> uio_pci_generic? >> > > I am not aware of any plans for it. > > Thanks for reviewing Flavio! > > >> fbl >> >> >> > --- >> > v5: >> > - use two appctl commands instead of a single one >> > - rebase to master >> > v4: >> > - fix typo in commit message >> > - remove unnecessary whitespace change in INSTALL.DPDK.md >> > v3: >> > - create dpdk_port_attach and dpdk_port_detach functions >> > - modify mutex locking order >> > v2: >> > - use rte_eth_dev_is_valid_port() to check if a port is valid >> > INSTALL.DPDK.md | 24 +++++++++++++ >> > NEWS | 1 + >> > lib/netdev-dpdk.c | 101 >> ++++++++++++++++++++++++++++++++++++++++++++++++++---- >> > 3 files changed, 119 insertions(+), 7 deletions(-) >> > >> > diff --git a/INSTALL.DPDK.md b/INSTALL.DPDK.md >> > index 93f92e4..8d5a0e0 100644 >> > --- a/INSTALL.DPDK.md >> > +++ b/INSTALL.DPDK.md >> > @@ -267,6 +267,29 @@ Using the DPDK with ovs-vswitchd: >> > For more details regarding egress-policer parameters please refer >> to the >> > vswitch.xml. >> > >> > +9. Port Hotplug >> > + >> > + Ovs supports port hotplugging, it allows to use ports that were not >> bound >> > + to DPDK when vswitchd was started. >> > + In order to attach a port, it has to be bound to DPDK using the >> > + dpdk_nic_bind.py script: >> > + >> > + `$DPDK_DIR/tools/dpdk_nic_bind.py --bind=igb_uio 0000:01:00.0` >> > + >> > + Then it can be attached to OVS: >> > + >> > + `ovs-appctl netdev-dpdk/port-attach 0000:01:00.0` >> > + >> > + At this point, the user can create a ovs port using the add-port >> command. >> > + >> > + It is also possible to detach a port from ovs, the user has to >> remove the >> > + port using the del-port command, then it can be detached using: >> > + >> > + `ovs-appctl netdev-dpdk/port-detach dpdk0` >> > + >> > + This feature is not supported by all the NICs, please refer to the >> > + [DPDK Port Hotplug Framework] in order to get more information. >> > + >> > Performance Tuning: >> > ------------------- >> > >> > @@ -999,3 +1022,4 @@ Please report problems to b...@openvswitch.org. >> > [INSTALL.md]:INSTALL.md >> > [DPDK Linux GSG]: >> http://www.dpdk.org/doc/guides/linux_gsg/build_dpdk.html#binding-and-unbinding-network-ports-to-from-the-igb-uioor-vfio-modules >> > [DPDK Docs]: http://dpdk.org/doc >> > +[DPDK Port Hotplug Framework]: >> http://dpdk.org/doc/guides/prog_guide/port_hotplug_framework.html >> > diff --git a/NEWS b/NEWS >> > index 4e81cad..d89d9a7 100644 >> > --- a/NEWS >> > +++ b/NEWS >> > @@ -32,6 +32,7 @@ Post-v2.5.0 >> > * DB entries have been added for many of the DPDK EAL command line >> > arguments. Additional arguments can be passed via the dpdk-extra >> > entry. >> > + * Port Hotplug is now supported. >> > - ovs-benchmark: This utility has been removed due to lack of use >> and >> > bitrot. >> > - ovs-appctl: >> > diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c >> > index af86d19..6159a60 100644 >> > --- a/lib/netdev-dpdk.c >> > +++ b/lib/netdev-dpdk.c >> > @@ -630,7 +630,7 @@ dpdk_eth_dev_init(struct netdev_dpdk *dev) >> OVS_REQUIRES(dpdk_mutex) >> > int diag; >> > int n_rxq, n_txq; >> > >> > - if (dev->port_id < 0 || dev->port_id >= rte_eth_dev_count()) { >> > + if (!rte_eth_dev_is_valid_port(dev->port_id)) { >> > return ENODEV; >> > } >> > >> > @@ -2118,6 +2118,83 @@ netdev_dpdk_set_admin_state(struct unixctl_conn >> *conn, int argc, >> > unixctl_command_reply(conn, "OK"); >> > } >> > >> > +static void >> > +netdev_dpdk_port_attach(struct unixctl_conn *conn, int argc OVS_UNUSED, >> > + const char *argv[], void *aux OVS_UNUSED) >> > +{ >> > + int ret; >> > + char response[128]; >> > + uint8_t port_id; >> > + >> > + ovs_mutex_lock(&dpdk_mutex); >> > + >> > + ret = rte_eth_dev_attach(argv[1], &port_id); >> > + if (ret < 0) { >> > + snprintf(response, sizeof(response), >> > + "Error attaching device '%s'", argv[1]); >> > + ovs_mutex_unlock(&dpdk_mutex); >> > + unixctl_command_reply_error(conn, response); >> > + return; >> > + } >> > + >> > + snprintf(response, sizeof(response), >> > + "Device '%s' has been attached as 'dpdk%d'", argv[1], >> port_id); >> > + >> > + ovs_mutex_unlock(&dpdk_mutex); >> > + unixctl_command_reply(conn, response); >> > +} >> > + >> > +static void >> > +netdev_dpdk_port_detach(struct unixctl_conn *conn, int argc OVS_UNUSED, >> > + const char *argv[], void *aux OVS_UNUSED) >> > +{ >> > + int ret; >> > + char response[128]; >> > + unsigned int parsed_port; >> > + uint8_t port_id; >> > + char devname[RTE_ETH_NAME_MAX_LEN]; >> > + >> > + ovs_mutex_lock(&dpdk_mutex); >> > + >> > + ret = dpdk_dev_parse_name(argv[1], "dpdk", &parsed_port); >> > + if (ret) { >> > + snprintf(response, sizeof(response), >> > + "'%s' is not a valid port", argv[1]); >> > + goto error; >> > + } >> > + >> > + port_id = parsed_port; >> > + >> > + struct netdev *netdev = netdev_from_name(argv[1]); >> > + if (netdev) { >> > + netdev_close(netdev); >> > + snprintf(response, sizeof(response), >> > + "Port '%s' is being used. Remove it before detaching", >> > + argv[1]); >> > + goto error; >> > + } >> > + >> > + rte_eth_dev_close(port_id); >> > + >> > + ret = rte_eth_dev_detach(port_id, devname); >> > + if (ret < 0) { >> > + snprintf(response, sizeof(response), >> > + "Port '%s' can not be detached", argv[1]); >> > + goto error; >> > + } >> > + >> > + snprintf(response, sizeof(response), >> > + "Port '%s' has been detached", argv[1]); >> > + >> > + ovs_mutex_unlock(&dpdk_mutex); >> > + unixctl_command_reply(conn, response); >> > + return; >> > + >> > +error: >> > + ovs_mutex_unlock(&dpdk_mutex); >> > + unixctl_command_reply_error(conn, response); >> > +} >> > + >> > /* >> > * Set virtqueue flags so that we do not receive interrupts. >> > */ >> > @@ -2404,6 +2481,15 @@ dpdk_common_init(void) >> > "[netdev] up|down", 1, 2, >> > netdev_dpdk_set_admin_state, NULL); >> > >> > + unixctl_command_register("netdev-dpdk/port-attach", >> > + "pci address of device", 1, 1, >> > + netdev_dpdk_port_attach, NULL); >> > + >> > + unixctl_command_register("netdev-dpdk/port-detach", >> > + "port", 1, 1, >> > + netdev_dpdk_port_detach, NULL); >> > + >> > + ovs_thread_create("dpdk_watchdog", dpdk_watchdog, NULL); >> > } >> > >> > /* Client Rings */ >> > @@ -2414,7 +2500,7 @@ dpdk_ring_create(const char dev_name[], unsigned >> int port_no, >> > { >> > struct dpdk_ring *ivshmem; >> > char ring_name[RTE_RING_NAMESIZE]; >> > - int err; >> > + int err, port_id; >> > >> > ivshmem = dpdk_rte_mzalloc(sizeof *ivshmem); >> > if (ivshmem == NULL) { >> > @@ -2448,19 +2534,20 @@ dpdk_ring_create(const char dev_name[], >> unsigned int port_no, >> > return ENOMEM; >> > } >> > >> > - err = rte_eth_from_rings(dev_name, &ivshmem->cring_rx, 1, >> > - &ivshmem->cring_tx, 1, SOCKET0); >> > + port_id = rte_eth_from_rings(dev_name, &ivshmem->cring_rx, 1, >> > + &ivshmem->cring_tx, 1, SOCKET0); >> > >> > - if (err < 0) { >> > + if (port_id < 0) { >> > rte_free(ivshmem); >> > return ENODEV; >> > } >> > >> > ivshmem->user_port_id = port_no; >> > - ivshmem->eth_port_id = rte_eth_dev_count() - 1; >> > + ivshmem->eth_port_id = port_id; >> > + *eth_port_id = port_id; >> > + >> > ovs_list_push_back(&dpdk_ring_list, &ivshmem->list_node); >> > >> > - *eth_port_id = ivshmem->eth_port_id; >> > return 0; >> > } >> > >> > -- >> > 1.9.1 >> > >> > _______________________________________________ >> > dev mailing list >> > dev@openvswitch.org >> > http://openvswitch.org/mailman/listinfo/dev >> >> -- >> fbl >> >> > _______________________________________________ dev mailing list dev@openvswitch.org http://openvswitch.org/mailman/listinfo/dev