On Mon, 2018-04-09 at 22:53 +0900, Moon-Sang Lee wrote: > I've configured a VM with KVM, and the VM is intended to run VPP with DPDK. > In particular, the VM is connected to one of VFs. (i.e. SR-IOV) > I can run DPDK sample applications,including l2fwd and l3fwd, in the VM, > therefore I guess VM is successfully connected to the outside-world(pktgen > server) via VFs. > > However, I cannot receive a packet when I run VPP/DPDK. > I can see TX packets from the VM on the opposite side, pktgen server, > but the VM does not receive any reply from pktgen server which reports RX/TX > packet count. > (i.e. arping/ping from VM arrives in pktgen, but the reply from pktgen is not > received in VM.) > I found some strange log messages regarding vpp launching as below. > > I appreciate for any comment. > Thanks in advance... > > - Host NIC: Intel 82599 10G NIC (i.e. VF binding with vfio-pci) > - VM: 1 socket 4 vCPU > - VPP: 18.04 > - DPDK binding: igb_uio It isn't clear to me who manages the PF in the host and how you created the VFs (kernel module or via DPDK binding)? Second, what do you mean by DPDK binding in the last line above?Is that what you have configured in startup.conf in the VM for VPP to be used? If so, what is the different between VF binding and DPDK binding in your short setup summary above? I'm confused by reading vfio-pci in one place and then igb_uio later on. Can you provide us with the startup.conf you have in the VM? Finally, if you are interested in using vfio-pci then you'll need to have the no-IOMMU enabled otherwise you can't use VFIO in the VM... probably, the easiest would be to use igb_uio everywhere... > root@xenial-vpp-frr:~# vpp -c /etc/vpp/startup.conf > vlib_plugin_early_init:359: plugin path /usr/lib/vpp_plugins > load_one_plugin:187: Loaded plugin: acl_plugin.so (Access Control Lists) > load_one_plugin:187: Loaded plugin: avf_plugin.so (Intel Adaptive Virtual > Function (AVF) Device Plugin) > load_one_plugin:189: Loaded plugin: cdp_plugin.so > load_one_plugin:187: Loaded plugin: dpdk_plugin.so (Data Plane Development Kit > (DPDK)) > load_one_plugin:187: Loaded plugin: flowprobe_plugin.so (Flow per Packet) > load_one_plugin:187: Loaded plugin: gbp_plugin.so (Group Based Policy) > load_one_plugin:187: Loaded plugin: gtpu_plugin.so (GTPv1-U) > load_one_plugin:187: Loaded plugin: igmp_plugin.so (IGMP messaging) > load_one_plugin:187: Loaded plugin: ila_plugin.so (Identifier-locator > addressing for IPv6) > load_one_plugin:187: Loaded plugin: ioam_plugin.so (Inbound OAM) > load_one_plugin:117: Plugin disabled (default): ixge_plugin.so > load_one_plugin:187: Loaded plugin: kubeproxy_plugin.so (kube-proxy data > plane) > load_one_plugin:187: Loaded plugin: l2e_plugin.so (L2 Emulation) > load_one_plugin:187: Loaded plugin: lacp_plugin.so (Link Aggregation Control > Protocol) > load_one_plugin:187: Loaded plugin: lb_plugin.so (Load Balancer) > load_one_plugin:187: Loaded plugin: memif_plugin.so (Packet Memory Interface > (experimetal)) > load_one_plugin:187: Loaded plugin: nat_plugin.so (Network Address > Translation) > load_one_plugin:187: Loaded plugin: pppoe_plugin.so (PPPoE) > load_one_plugin:187: Loaded plugin: router.so (router) > load_one_plugin:187: Loaded plugin: srv6ad_plugin.so (Dynamic SRv6 proxy) > load_one_plugin:187: Loaded plugin: srv6am_plugin.so (Masquerading SRv6 proxy) > load_one_plugin:187: Loaded plugin: srv6as_plugin.so (Static SRv6 proxy) > load_one_plugin:187: Loaded plugin: stn_plugin.so (VPP Steals the NIC for > Container integration) > load_one_plugin:187: Loaded plugin: tlsmbedtls_plugin.so (mbedtls based TLS > Engine) > load_one_plugin:187: Loaded plugin: tlsopenssl_plugin.so (openssl based TLS > Engine) > load_one_plugin:67: Loaded plugin: > /usr/lib/vpp_api_test_plugins/flowprobe_test_plugin.so > load_one_plugin:67: Loaded plugin: > /usr/lib/vpp_api_test_plugins/dpdk_test_plugin.so > load_one_plugin:67: Loaded plugin: > /usr/lib/vpp_api_test_plugins/ioam_export_test_plugin.so > load_one_plugin:67: Loaded plugin: > /usr/lib/vpp_api_test_plugins/cdp_test_plugin.so > load_one_plugin:67: Loaded plugin: > /usr/lib/vpp_api_test_plugins/kubeproxy_test_plugin.so > load_one_plugin:67: Loaded plugin: > /usr/lib/vpp_api_test_plugins/ioam_vxlan_gpe_test_plugin.so > load_one_plugin:67: Loaded plugin: > /usr/lib/vpp_api_test_plugins/stn_test_plugin.so > load_one_plugin:67: Loaded plugin: > /usr/lib/vpp_api_test_plugins/acl_test_plugin.so > load_one_plugin:67: Loaded plugin: > /usr/lib/vpp_api_test_plugins/lb_test_plugin.so > load_one_plugin:67: Loaded plugin: > /usr/lib/vpp_api_test_plugins/lacp_test_plugin.so > load_one_plugin:67: Loaded plugin: > /usr/lib/vpp_api_test_plugins/ioam_pot_test_plugin.so > load_one_plugin:67: Loaded plugin: > /usr/lib/vpp_api_test_plugins/pppoe_test_plugin.so > load_one_plugin:67: Loaded plugin: > /usr/lib/vpp_api_test_plugins/udp_ping_test_plugin.so > load_one_plugin:67: Loaded plugin: > /usr/lib/vpp_api_test_plugins/vxlan_gpe_ioam_export_test_plugin.so > load_one_plugin:67: Loaded plugin: > /usr/lib/vpp_api_test_plugins/memif_test_plugin.so > load_one_plugin:67: Loaded plugin: > /usr/lib/vpp_api_test_plugins/gtpu_test_plugin.so > load_one_plugin:67: Loaded plugin: > /usr/lib/vpp_api_test_plugins/ioam_trace_test_plugin.so > load_one_plugin:67: Loaded plugin: > /usr/lib/vpp_api_test_plugins/nat_test_plugin.so > dpdk_config:1271: EAL init args: -c 1 -n 4 --huge-dir /run/vpp/hugepages -- > file-prefix vpp -w 0000:00:06.0 -w 0000:00:07.0 --master-lcore 0 --socket-mem > 512 > EAL: No free hugepages reported in hugepages-1048576kB > EAL: VFIO support initialized > EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable > clock cycles ! > EAL: Invalid NUMA socket, default to 0 > EAL: Invalid NUMA socket, default to 0 > DPDK physical memory layout: > Segment 0: IOVA:0x33400000, len:25165824, virt:0x7ff739c00000, socket_id:0, > hugepage_sz:2097152, nchannel:0, nrank:0 > Segment 1: IOVA:0x34e00000, len:2097152, virt:0x7ff739800000, socket_id:0, > hugepage_sz:2097152, nchannel:0, nrank:0 > Segment 2: IOVA:0x87800000, len:509607936, virt:0x7ff707e00000, socket_id:0, > hugepage_sz:2097152, nchannel:0, nrank:0 > dpdk_pool_create:550: ioctl(VFIO_IOMMU_MAP_DMA) pool 'dpdk_mbuf_pool_socket0': > Invalid argument (errno 22) > 0: dpdk_ipsec_process:1012: not enough DPDK crypto resources, default to > OpenSSL > 0: dpdk_lib_init:234: DPDK drivers found 2 ports... > _______ _ _ _____ ___ > __/ __/ _ \ (_)__ | | / / _ \/ _ \ > _/ _// // / / / _ \ | |/ / ___/ ___/ > /_/ /____(_)_/\___/ |___/_/ /_/ > > DBGvpp# > > > -- Marco V
SUSE LINUX GmbH | GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg) Maxfeldstr. 5, D-90409, Nürnberg
