Hi all, I'm using ovs-dpdk version 2.13 in openstack(openstack is deploy by kolla-ansible(https://docs.openstack.org/kolla-ansible/latest/)).
After I reboot hipervisor, I found high cpu usage of pmd thread in ovs-dpdk. But there is no large log or large packets, how to know what's the problem? I want to use perf like this: https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/10/html/ovs-dpdk_end_to_end_troubleshooting_guide/troubleshoot_ovs_dpdk_pmd_cpu_usage_with_perf_and_collect_and_send_the_troubleshooting_data But it's hang there. So I want to know, if it's my config error? Like there should be some other config in Open_vSwitch.other_config of ovsdb? Thank you!!! ``` No more log: [root@host01 ~]# tailf /var/log/kolla/openvswitch/ovs ovsdb-server.log ovs-vswitchd.log [root@host01 ~]# tailf /var/log/kolla/openvswitch/ovs-vswitchd.log 2021-10-09T02:41:16.175Z|00351|dpif_netdev|WARN|There's no available (non-isolated) pmd thread on numa node 0. Queue 0 on port 'enp5s0f0' will be assigned to the pmd on core 1 (numa node 1). Expect reduced performance. 2021-10-09T02:41:16.177Z|00352|bridge|INFO|bridge br_mgmt: added interface enp5s0f1.1001 on port 1 2021-10-09T02:41:16.181Z|00353|bridge|INFO|bridge br_mgmt: using datapath ID 0000043f72a49981 2021-10-09T02:41:16.181Z|00354|rconn|INFO|br_mgmt<->tcp:127.0.0.1:6633: disconnecting 2021-10-09T02:41:16.187Z|00355|netdev_linux|WARN|error sending Ethernet packet on enp5s0f1.1001: Network is down 2021-10-09T02:41:17.210Z|00356|rconn|INFO|br_mgmt<->tcp:127.0.0.1:6633: connecting... 2021-10-09T02:41:18.035Z|00357|rconn|INFO|br_mgmt<->tcp:127.0.0.1:6633: connected ^C no packet on this NIC: [root@host01 ~]# tcpdump -i enp5s0f0 -c 100 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on enp5s0f0, link-type EN10MB (Ethernet), capture size 262144 bytes ^C 0 packets captured 0 packets received by filter 0 packets dropped by kernel top result: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 23387 root 20 0 256.9g 467624 24984 S 96.1 0.7 882:43.45 ovs-vswitchd top -H -p 23387 result: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 23444 root 20 0 256.9g 467624 24984 R 95.7 0.7 883:35.70 pmd-c01/id:7 (ovsdpdk-db)[root@host01 openvswitch-2.13.4]# ovs-vsctl show | grep -C 4 enp5s0f0 Port phy-br_data Interface phy-br_data type: patch options: {peer=int-br_data} Port enp5s0f0 (This is dpdk type netdev, which is PMD by pmd-c01) Interface enp5s0f0 type: dpdk options: {dpdk-devargs="0000:05:00.0"} Port br_data Interface br_data (ovsdpdk-db)[root@host01 openvswitch-2.13.4]# ovs-vsctl list Open_vSwitch _uuid : 427f600b-7a06-46e9-b273-5e63e08b1c72 bridges : [606f95bb-0637-4df9-809a-6b85f618ac89, ce70bf43-ebad-4540-a7e3-e22a6e82fc4d, eae3413a-c859-44b3-a0dc-3b11c1f60a77, fa0eeef2-c15f-4f49-8ad6-72deb0d2b120] cur_cfg : 90 datapath_types : [netdev, system] datapaths : {} db_version : [] dpdk_initialized : true dpdk_version : "DPDK 19.11.8" external_ids : {} iface_types : [dpdk, dpdkr, dpdkvhostuser, dpdkvhostuserclient, erspan, geneve, gre, internal, ip6erspan, ip6gre, lisp, patch, stt, system, tap, vxlan] manager_options : [65e211d5-a792-4f28-9fab-4eb42034acaf] next_cfg : 90 other_config : {dpdk-extra=" --proc-type primary ", dpdk-hugepage-dir="/dev/hugepages", dpdk-init=True, dpdk-lcore-mask="0x1", dpdk-mem-channels="4", dpdk-socket-mem="1024", pmd-cpu-mask="0x2"} ovs_version : [] ssl : [] statistics : {} system_type : [] system_version : [] ``` ---- Simon Jones _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
