Hello, I am trying to use vpp with 4 worker threads. Client is sending packet to server through vpp. This packet comes to vpp, and gets picked up from a VF. VPP sends this packet to the another user-space application over a tapcli interface. The user-space application processes the packet, and then sends it back to vpp. Vpp forwards the packet to the server.
I am using vpp with 4 worker threads. Without the traffic, four worker threads are all showing ~100% cpu usage (which is expected). However, with traffic, the vpp main thread is also showing ~100% cpu usage. I do not want vpp main thread to reach 100% cpu. With the limited knowledge that I have on vpp, it seems like main thread is doing lots of processing in the tapcli-rx node. Is there a way to keep vpp main relatively free? Is tap interface the correct choice to communicate between vpp and another user-space application? If not, what should be used? I have tried this on both vpp 18.07 and 18.10. vpp# show run Thread 0 vpp_main (lcore 2) Time 916.9, average vectors/node 40.16, last 128 main loops 0.00 per node 0.00 vector rates in 1.3178e5, out 1.3178e5, drop 1.3087e-2, punt 0.0000e0 Name State Calls Vectors Suspends Clocks Vectors/Call VirtualFunctionEthernet3/10/1- active 3008704 120840447 0 1.36e1 40.16 VirtualFunctionEthernet3/10/1- active 3008704 120840447 0 1.02e2 40.16 acl-plugin-fa-cleaner-process event wait 0 0 1 1.85e4 0.00 admin-up-down-process event wait 0 0 1 1.20e3 0.00 api-rx-from-ring any wait 0 0 50 7.80e4 0.00 arp-input active 1 1 0 2.39e6 1.00 avf-process event wait 0 0 1 2.14e3 0.00 bfd-process event wait 0 0 1 1.84e3 0.00 bond-process event wait 0 0 1 1.44e3 0.00 cdp-process any wait 0 0 1 1.38e7 0.00 dhcp-client-process any wait 0 0 10 1.02e4 0.00 dhcp6-client-cp-process any wait 0 0 1 1.07e3 0.00 dhcp6-pd-client-cp-process any wait 0 0 1 9.28e2 0.00 dhcp6-pd-reply-publisher-proce event wait 0 0 1 8.88e2 0.00 dhcp6-reply-publisher-process event wait 0 0 1 1.02e3 0.00 dns-resolver-process any wait 0 0 1 2.18e3 0.00 dpdk-ipsec-process done 1 0 0 1.29e5 0.00 dpdk-process any wait 0 0 306 5.73e4 0.00 error-drop active 11 12 0 5.73e3 1.09 ethernet-input active 3008710 120840454 0 3.78e1 40.16 fib-walk any wait 0 0 459 3.03e3 0.00 flow-report-process any wait 0 0 1 6.92e2 0.00 flowprobe-timer-process any wait 0 0 1 3.09e3 0.00 igmp-timer-process event wait 0 0 1 2.18e3 0.00 ikev2-manager-process any wait 0 0 916 2.46e3 0.00 ioam-export-process any wait 0 0 1 1.10e3 0.00 ip-neighbor-scan-process any wait 0 0 16 7.03e3 0.00 ip-route-resolver-process any wait 0 0 10 8.13e3 0.00 ip4-glean active 2 2 0 9.34e3 1.00 ip4-input active 3008704 120840447 0 4.03e1 40.16 ip4-lookup active 3008704 120840447 0 3.67e1 40.16 ip4-reassembly-expire-walk any wait 0 0 92 3.96e3 0.00 ip4-rewrite active 3008702 120840445 0 3.08e1 40.16 ip6-icmp-neighbor-discovery-ev any wait 0 0 916 2.00e3 0.00 ip6-input active 5 6 0 1.17e3 1.20 ip6-not-enabled active 5 6 0 5.45e2 1.20 ip6-reassembly-expire-walk any wait 0 0 92 4.30e3 0.00 l2fib-mac-age-scanner-process event wait 0 0 1 1.78e3 0.00 lacp-process event wait 0 0 1 1.41e7 0.00 lisp-retry-service any wait 0 0 459 4.11e3 0.00 lldp-process event wait 0 0 1 1.31e7 0.00 memif-process event wait 0 0 1 4.28e3 0.00 nat-det-expire-walk done 1 0 0 1.23e3 0.00 nat64-expire-walk event wait 0 0 1 1.46e4 0.00 rd-cp-process any wait 0 0 1 9.16e2 0.00 send-dhcp6-client-message-proc any wait 0 0 1 2.88e3 0.00 send-dhcp6-pd-client-message-p any wait 0 0 1 1.02e3 0.00 send-garp-na-process event wait 0 0 1 2.88e4 0.00 send-rs-process any wait 0 0 1 1.24e3 0.00 startup-config-process done 1 0 1 3.75e3 0.00 tapcli-rx interrupt wa 5382120 120840506 0 2.42e3 22.45 udp-ping-process any wait 0 0 1 5.70e3 0.00 unix-cli-local:6 active 6 0 757 3.46e15 0.00 unix-epoll-input polling 16312904 0 0 6.94e4 0.00 vhost-user-process any wait 0 0 1 1.32e4 0.00 vhost-user-send-interrupt-proc any wait 0 0 1 1.03e3 0.00 vpe-link-state-process event wait 0 0 3 4.95e3 0.00 vpe-oam-process any wait 0 0 450 2.86e3 0.00 vxlan-gpe-ioam-export-process any wait 0 0 1 8.76e2 0.00 wildcard-ip4-arp-publisher-pro event wait 0 0 1 1.09e3 0.00 --------------- Thread 1 vpp_wk_0 (lcore 3) Time 916.9, average vectors/node 1.43, last 128 main loops 0.00 per node 0.00 vector rates in 3.2809e4, out 3.2809e4, drop 5.6599e-1, punt 0.0000e0 Name State Calls Vectors Suspends Clocks Vectors/Call arp-input active 518 518 0 7.74e3 1.00 dpdk-input polling 3205687180 30084768 0 4.31e4 0.00 error-drop active 519 519 0 1.28e3 1.00 ethernet-input active 518 518 0 1.73e3 1.00 ip4-glean active 1 1 0 6.62e4 1.00 ip4-input-no-checksum active 20966739 30084250 0 3.64e2 1.43 ip4-lookup active 20966739 30084250 0 2.46e2 1.43 ip4-rewrite active 20966738 30084249 0 1.93e2 1.43 tapcli-0-output active 20966739 30084250 0 2.03e2 1.43 tapcli-0-tx active 20966739 30084250 0 1.29e4 1.43 --------------- Thread 2 vpp_wk_1 (lcore 4) Time 916.9, average vectors/node 1.45, last 128 main loops 0.00 per node 0.00 vector rates in 3.2819e4, out 3.2819e4, drop 0.0000e0, punt 0.0000e0 Name State Calls Vectors Suspends Clocks Vectors/Call dpdk-input polling 3062149112 30094113 0 4.24e4 0.00 ip4-input-no-checksum active 20687846 30094113 0 3.67e2 1.45 ip4-lookup active 20687846 30094113 0 2.59e2 1.45 ip4-rewrite active 20687846 30094113 0 2.03e2 1.45 tapcli-0-output active 20687846 30094113 0 2.11e2 1.45 tapcli-0-tx active 20687846 30094113 0 1.31e4 1.45 --------------- Thread 3 vpp_wk_2 (lcore 5) Time 916.9, average vectors/node 1.46, last 128 main loops 0.00 per node 0.00 vector rates in 3.2774e4, out 3.2774e4, drop 2.4428e-1, punt 0.0000e0 Name State Calls Vectors Suspends Clocks Vectors/Call dpdk-input polling 2997474079 30052436 0 4.33e4 .01 error-drop active 224 224 0 1.57e3 1.00 ip4-drop active 224 224 0 8.69e2 1.00 ip4-input-no-checksum active 20609931 30052436 0 3.74e2 1.46 ip4-lookup active 20609931 30052436 0 2.64e2 1.46 ip4-rewrite active 20609737 30052212 0 2.01e2 1.46 tapcli-0-output active 20609737 30052212 0 2.15e2 1.46 tapcli-0-tx active 20609737 30052212 0 1.31e4 1.46 --------------- Thread 4 vpp_wk_3 (lcore 6) Time 916.9, average vectors/node 1.45, last 128 main loops 0.00 per node 0.00 vector rates in 3.2811e4, out 3.2811e4, drop 0.0000e0, punt 0.0000e0 Name State Calls Vectors Suspends Clocks Vectors/Call dpdk-input polling 3135905643 30086296 0 4.30e4 0.00 ip4-input-no-checksum active 20714610 30086296 0 3.74e2 1.45 ip4-lookup active 20714610 30086296 0 2.55e2 1.45 ip4-rewrite active 20714610 30086296 0 1.97e2 1.45 tapcli-0-output active 20714610 30086296 0 2.04e2 1.45 tapcli-0-tx active 20714610 30086296 0 1.31e4 1.45 Here is one of the packets traced on vpp main thread. vpp# show trace ------------------- Start of thread 0 vpp_main ------------------- Packet 1 00:11:26:605715: tapcli-rx tapcli-0 00:11:26:605725: ethernet-input IP4: c2:aa:32:4f:d8:1c -> 02:fe:d9:0f:5c:f4 00:11:26:605741: ip4-input TCP: 10.0.1.1 -> 172.18.60.74 tos 0x00, ttl 64, length 60, checksum 0x475f fragment id 0x0000, flags DONT_FRAGMENT TCP: 82 -> 60818 seq. 0xcbbfa586 ack 0x36bb48c6 flags 0x12 SYN ACK, tcp header: 40 bytes window 28960, checksum 0xab2b 00:11:26:605748: ip4-lookup fib 0 dpo-idx 3 flow hash: 0x00000000 TCP: 10.0.1.1 -> 172.18.60.74 tos 0x00, ttl 64, length 60, checksum 0x475f fragment id 0x0000, flags DONT_FRAGMENT TCP: 82 -> 60818 seq. 0xcbbfa586 ack 0x36bb48c6 flags 0x12 SYN ACK, tcp header: 40 bytes window 28960, checksum 0xab2b 00:11:26:605754: ip4-rewrite tx_sw_if_index 1 dpo-idx 3 : ipv4 via 172.18.60.74 VirtualFunctionEthernet3/10/1: mtu:9000 ac1f6b17f379da78bd634ef70800 flow hash: 0x00000000 00000000: ac1f6b17f379da78bd634ef708004500003c000040003f06485f0a000101ac12 00000020: 3c4a0052ed92cbbfa58636bb48c6a0127120ab2b0000020405b40402 00:11:26:605755: VirtualFunctionEthernet3/10/1-output VirtualFunctionEthernet3/10/1 IP4: da:78:bd:63:4e:f7 -> ac:1f:6b:17:f3:79 TCP: 10.0.1.1 -> 172.18.60.74 tos 0x00, ttl 63, length 60, checksum 0x485f fragment id 0x0000, flags DONT_FRAGMENT TCP: 82 -> 60818 seq. 0xcbbfa586 ack 0x36bb48c6 flags 0x12 SYN ACK, tcp header: 40 bytes window 28960, checksum 0xab2b 00:11:26:605758: VirtualFunctionEthernet3/10/1-tx VirtualFunctionEthernet3/10/1 tx queue 0 buffer 0x23e5a0: current data 0, length 74, free-list 0, clone-count 0, totlen-nifb 0, trace 0x0 l2-hdr-offset 0 l3-hdr-offset 14 PKT MBUF: port 65535, nb_segs 1, pkt_len 74 buf_len 2176, data_len 74, ol_flags 0x0, data_off 128, phys_addr 0xe7996880 packet_type 0x0 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 rss 0x0 fdir.hi 0x0 fdir.lo 0x0 IP4: da:78:bd:63:4e:f7 -> ac:1f:6b:17:f3:79 TCP: 10.0.1.1 -> 172.18.60.74 tos 0x00, ttl 63, length 60, checksum 0x485f fragment id 0x0000, flags DONT_FRAGMENT TCP: 82 -> 60818 seq. 0xcbbfa586 ack 0x36bb48c6 flags 0x12 SYN ACK, tcp header: 40 bytes window 28960, checksum 0xab2b
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#11999): https://lists.fd.io/g/vpp-dev/message/11999 Mute This Topic: https://lists.fd.io/mt/29534956/21656 Mute #vpp: https://lists.fd.io/mk?hashtag=vpp&subid=1480452 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
