The patch series introduces AF_XDP support for OVS netdev. AF_XDP is a new address family working together with eBPF. In short, a socket with AF_XDP family can receive and send packets from an eBPF/XDP program attached to the netdev. For more details about AF_XDP, please see linux kernel's Documentation/networking/af_xdp.rst
OVS has a couple of netdev types, i.e., system, tap, or internal. The patch first adds a new netdev types called "afxdp", and implement its configuration, packet reception, and transmit functions. Since the AF_XDP socket, xsk, operates in userspace, once ovs-vswitchd receives packets from xsk, the proposed architecture re-uses the existing userspace dpif-netdev datapath. As a result, most of the packet processing happens at the userspace instead of linux kernel. Architecure =========== _ | +-------------------+ | | ovs-vswitchd |<-->ovsdb-server | +-------------------+ | | ofproto |<-->OpenFlow controllers | +--------+-+--------+ | | netdev | |ofproto-| userspace | +--------+ | dpif | | | netdev | +--------+ | |provider| | dpif | | +---||---+ +--------+ | || | dpif- | | || | netdev | |_ || +--------+ || _ +---||-----+--------+ | | af_xdp prog + | kernel | | xsk_map | |_ +--------||---------+ || physical NIC To simply start, create a ovs userspace bridge using dpif-netdev by setting the datapath_type to netdev: # ovs-vsctl -- add-br br0 -- set Bridge br0 datapath_type=netdev And attach a linux netdev with type afxdp: # ovs-vsctl add-port br0 afxdp-p0 -- \ set interface afxdp-p0 type="afxdp" Most of the implementation follows the AF_XDP sample code in Linux kernel under samples/bpf/xdpsock_user.c. Configuration ============= When a new afxdp netdev is added to OVS, the patch does the following configuration 1) attach the afxdp program and map to the netdev (see bpf/xdp.h) 2) create an AF_XDP socket (XSK) for thie netdev 3) allocate a virtual contiguous memory region, called umem, and register this memory to the XSK 4) setup the rx/tx ring, and umem's fill/completion ring Packet Flow =========== Currently, the af_xdp program loaded to the netdev does nothing but simply forwards the packet to queue id 0. The patch simplifies the buffer/ring management by introducing a copy from umem to ovs's internal buffer, when receiving a packet. And when sending the packet out to another netdev, copying the packet to the netdev's umem. An AF_XDP packet forwarding from one netdev (ovs-p0) to another netdev (ovs-p1) goes through the following path: 1) xdp program at ovs-p0 copies packet to kernel (SKB_MODE) 2) kernel maps the packet to userspace umem 3) ovs-vswitchd receive the packet from ovs-p0, copy to internal packet buffer 4) ovs-vswitchd copies the pachet to umem of ovs-p1, kick_tx 5) kernel copies the packet from umem to ovs-p1 tx queue Since the total number of copies between two ports is 4, the performance will be bad so I don't want to do it. Hopefully by using AF_XDP zero copy mode, 1) and 5) will be removed and in ovs-vswitchd, it's possible to combine the 3) and 4) to only one copy. So the best case will be one copy between two netdev. Test Framework ============== # make check-afxdp will kick start two end-to-end tests using veth peer and namespaces: AFXDP netdev datapath-sanity 1: datapath - ping between two ports ok 2: datapath - http between two ports ok The patch series is based on the ovs-ebpf implementaion. A copy is put at: https://github.com/williamtu/ovs-ebpf/ branch afxdp-v1 William Tu (3): afxdp: add ebpf code for afxdp and xskmap. netdev-linux: add new netdev type afxdp. tests: add afxdp test cases. acinclude.m4 | 1 + bpf/api.h | 6 + bpf/helpers.h | 2 + bpf/maps.h | 12 + bpf/xdp.h | 34 +- lib/automake.mk | 3 +- lib/bpf.c | 41 ++- lib/bpf.h | 6 +- lib/dpif-netdev.c | 74 +++- lib/if_xdp.h | 79 +++++ lib/netdev-dummy.c | 1 + lib/netdev-linux.c | 741 +++++++++++++++++++++++++++++++++++++++- lib/netdev-provider.h | 2 + lib/netdev-vport.c | 4 + lib/netdev.c | 11 + lib/netdev.h | 1 + tests/automake.mk | 17 + tests/ofproto-macros.at | 1 + tests/system-afxdp-macros.at | 148 ++++++++ tests/system-afxdp-testsuite.at | 25 ++ tests/system-afxdp-traffic.at | 38 +++ vswitchd/bridge.c | 1 + 22 files changed, 1228 insertions(+), 20 deletions(-) create mode 100644 lib/if_xdp.h create mode 100644 tests/system-afxdp-macros.at create mode 100644 tests/system-afxdp-testsuite.at create mode 100644 tests/system-afxdp-traffic.at -- 2.7.4 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1456): https://lists.iovisor.org/g/iovisor-dev/message/1456 Mute This Topic: https://lists.iovisor.org/mt/24652715/21656 Group Owner: iovisor-dev+ow...@lists.iovisor.org Unsubscribe: https://lists.iovisor.org/g/iovisor-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-