Hi Vijay, That’s probably because packets are hitting an actual error and it seems the listen node is not reporting anything but syns received. Here’s a patch that might help [1]. It might not cherry-pick cleanly on 21.06.
Regards, Florin [1] https://gerrit.fd.io/r/c/vpp/+/35654 > On Mar 15, 2022, at 7:54 PM, Vijay Kumar <vjkumar2...@gmail.com> wrote: > > Hi Florin, > > I have the output of show node counters (show errors) taken on both 20.05 and > 21.06 vpp. In 21.06 I don't see any counters in tcp4-listen or tcp4-output > etc. > > Please let me know why SYN rcvd counter itself is not incremented but in the > earlier reply I already pasted the show trace ouput where we saw the SYN pkt > was landed on tcp4-litsen node. > > VPP 20.05 > =========== > pp# show node counters > Count Node Reason > 3 memif-input not ip packet > 10259 an_ppe_wfectrl wfectrl packets received > 10259 an_ppe_wfectrl wfectrl replies sent > 1 an_ppe_wfectrl wfectrl packet processing > failed > 8123 an_ppe_wfectrl session stat request > received > 1 an_ppe_wfectrl service construct config > request received > 1 an_ppe_wfectrl service construct config > request success > 1 an_ppe_wfectrl service config request > received > 1 an_ppe_wfectrl service config request > success > 1686 an_ppe_wfectrl dpi stats request received > 1686 an_ppe_wfectrl dpi stats request success > 70 an_ppe_wfectrl nat stats request received > 70 an_ppe_wfectrl nat stats request success > 70 an_ppe_wfectrl vpp stats request received > 70 an_ppe_wfectrl vpp stats request success > 1 an_ppe_wfectrl udp tunnel resource block > add request received > 1 an_ppe_wfectrl udp tunnel resource block > add request success > 1 an_ppe_wfectrl l3rm resource block update > request received > 1 an_ppe_wfectrl l3rm resource block update > request success > 1 an_ppe_wfectrl ue registration request > received > 17 an_ppe_wfectrl ike msg received from ikemgr > 17 an_ppe_wfectrl ike msg send to network > success > 3 an_ppe_wfectrl ipsec sa install msg > received from ikemgr > 3 an_ppe_wfectrl ipsec sa install msg > processed successfully > 1 an_ppe_vppctrl_input Success in insert in > ueidentifier > 1 an_ppe_vppctrl_input Success in insert in > uecontext > 2 an_ppe_vppctrl_input Downlink message sent to UE > successfully > 1 an_ppe_vppctrl_input TCP ESTABLISHMENT message > received > 1 an_ppe_vppctrl_input UPLINK NAS message received > 1 an_ppe_vppctrl_input NAS TCP CLOSE message > received > 7 an_ppe_router_input packets received in rtr > 7 an_ppe_router_input packets dropped to host. no > tacptcp session found > 17 an-ppe-isakmp4-output Total IKEV4 packets > dispatched to network > 17 an-ppe-isakmpmgr-input packets processed by > isakmpmgr input plugin > 17 an-ppe-isakmpmgr-input Received IKE packets > 17 an-ppe-isakmpmgr-input Successfully sent ike > message to Session Manager > 5 an-ppe-isakmpmgr-input Received ike exchange AUTH > packet > 4 an-ppe-isakmpmgr-input Received ike exchange > CREATE CHILD SA packet > 1 an-ppe-isakmpmgr-input Received ike exchange SA > INIT packet > 7 an-ppe-isakmpmgr-input Received ike exchange > INFORMATIONAL packet > 5 arp-reply ARP replies sent > 4 session-queue Packets transmitted > 17 ip4-udp-lookup No error > 1 tcp4-listen SYNs received > 2 tcp4-rcv-process Pure ACKs received > 1 tcp4-established Packets pushed into rx fifo > 2 tcp4-established Pure ACKs received > 1 tcp4-established FINs received > 5 tcp4-output Packets sent > 7 esp4-decrypt-tun ESP pkts received > 5 esp4-encrypt-tun ESP pkts received > 5 esp4-encrypt-tun total control pkts received > 7 ipsec4-tun-input good packets received > 2 ip4-local unknown ip protocol > 5 ethernet-input unknown vlan > vpp# > vpp# > > > > > > VPP 21.06 output > =============== > vpp# > vpp# show version > vpp v21.06.0-3~g2a4af2b5d-dirty built by an-vijay_kumar on 500aaad2d090 at > 2022-03-15T01:16:27 > vpp# > vpp# > vpp# show node counters > Count Node Reason > Severity > 1 null-node blackholed > packets error > 6690 an_ppe_wfectrl wfectrl packets > received error > 6690 an_ppe_wfectrl wfectrl > replies sent error > 1 an_ppe_wfectrl wfectrl packet > processing failed error > 5329 an_ppe_wfectrl session stat > request received error > 1 an_ppe_wfectrl service construct config > request received error > 1 an_ppe_wfectrl service construct config > request success error > 1 an_ppe_wfectrl service config > request received error > 1 an_ppe_wfectrl service config > request success error > 1068 an_ppe_wfectrl dpi stats request > received error > 1068 an_ppe_wfectrl dpi stats request > success error > 44 an_ppe_wfectrl nat stats request > received error > 44 an_ppe_wfectrl nat stats request > success error > 44 an_ppe_wfectrl vpp stats request > received error > 44 an_ppe_wfectrl vpp stats request > success error > 1 an_ppe_wfectrl udp tunnel resource block > add request received error > 1 an_ppe_wfectrl udp tunnel resource block > add request success error > 1 an_ppe_wfectrl l3rm resource block update > request received error > 1 an_ppe_wfectrl l3rm resource block > update request success error > 1 an_ppe_wfectrl ue registration > request received error > 1 an_ppe_wfectrl ue deregistration > request received error > 14 an_ppe_wfectrl ike msg received > from ikemgr error > 14 an_ppe_wfectrl ike msg send to > network success error > 3 an_ppe_wfectrl ipsec sa install msg > received from ikemgr error > 3 an_ppe_wfectrl ipsec sa delete msg > received from ikemgr error > 3 an_ppe_wfectrl ipsec sa install msg > processed successfully error > 3 an_ppe_wfectrl ipsec sa delete msg > processed successfully error > 1 an_ppe_vppctrl_input Success in delete in > ueidentifier error > 1 an_ppe_vppctrl_input Success in insert in > ueidentifier error > 1 an_ppe_vppctrl_input Success in insert > in uecontext error > 1 an_ppe_vppctrl_input Success in delete > in uecontext error > 5 an_ppe_router_input packets received > in rtr error > 5 an_ppe_router_input packets dropped to host. no > tacptcp session found error > 14 an-ppe-isakmp4-output Total IKEV4 packets > dispatched to network error > 14 an-ppe-isakmpmgr-input packets processed by > isakmpmgr input plugin error > 14 an-ppe-isakmpmgr-input Received IKE > packets error > 14 an-ppe-isakmpmgr-input Successfully sent ike > message to Session Manager error > 5 an-ppe-isakmpmgr-input Received ike exchange > AUTH packet error > 4 an-ppe-isakmpmgr-input Received ike exchange > CREATE CHILD SA packet error > 1 an-ppe-isakmpmgr-input Received ike exchange > SA INIT packet error > 4 an-ppe-isakmpmgr-input Received ike exchange > INFORMATIONAL packet error > 3 arp-reply ARP replies > sent error > 14 ip4-udp-lookup No error > error > 5 esp4-decrypt-tun ESP pkts > received error > 5 ipsec4-tun-input good packets > received error > 1 ip4-input ip4 ttl <= > 1 error > 1 ip4-icmp-error hop limit exceeded > response sent error > 27628 ethernet-input unknown > vlan error > vpp# > vpp# > > > > > On Wed, Mar 16, 2022 at 6:52 AM Florin Coras <fcoras.li...@gmail.com > <mailto:fcoras.li...@gmail.com>> wrote: > Hi Vijay, > > You won’t see the syn/syn-acks in traces because of the way they are > generated. Nonetheless, you can verify that the syns were received with “show > error” and check that the syn-acks were actually generated with a pcap trace > rxtx <output interface> > > Previously tracing worked because buffers were re-used and incoming packets > were directly sent to output. More recently we’ve started dispatching > syn-acks through the session layer in order to minimize size of tx bursts per > dispatch. > > Regards, > Florin > >> On Mar 15, 2022, at 6:08 PM, Vijay Kumar <vjkumar2...@gmail.com >> <mailto:vjkumar2...@gmail.com>> wrote: >> >> Hi Florin, >> >> Thanks for the quick response. >> >> It means it is expected to see the SYN pkts getting dropped after it is >> terminated but I was not able to see any SYN-ACK going out of VPP. >> >> In both, "show trace" cmd output and also the pcap taken at dspatch graph >> node level, I was not able to see the SYN-ACK going out. The route to the >> SYN-ACK destination (which is the original source that started SYN) is also >> present in the ip fib output. The configuration is the same that was working >> fine for me in vp 20.05. >> >> Is there anything that I can look at for the SYN-ACK not sending issue? >> >> >> >> On Wed, 16 Mar 2022, 00:40 Florin Coras, <fcoras.li...@gmail.com >> <mailto:fcoras.li...@gmail.com>> wrote: >> Hi Vijay, >> >> I see an_ppe_router_input forwards syns to tcp-input and those packets are >> delivered to tcp-listen, i.e., you’ve created a listener that’s matched by >> the incoming traffic. >> >> The thing to keep in mind is that tcp terminates incoming flows and it does >> not reuse buffers. That is, the syn hits the listen node and a syn-ack is >> programmed for this new connection that still needs to complete the >> handshake. The original syn packet is discarded and therefore you see it as >> a drop. >> >> Regards, >> Florin >> >>> On Mar 15, 2022, at 3:50 AM, Vijay Kumar <vjkumar2...@gmail.com >>> <mailto:vjkumar2...@gmail.com>> wrote: >>> >>> The is the output of show trace and show interface >>> ============================================ >>> >>> Packet 36 >>> >>> 00:03:26:875694: dpdk-input >>> VirtualFuncEthernet0/7/0 rx queue 0 >>> buffer 0x4c1b21: current data 0, length 138, buffer-pool 0, ref-count 1, >>> totlen-nifb 0, trace handle 0x1000023 >>> ext-hdr-valid >>> l4-cksum-computed l4-cksum-correct >>> PKT MBUF: port 0, nb_segs 1, pkt_len 138 >>> buf_len 2176, data_len 138, ol_flags 0x182, data_off 128, phys_addr >>> 0xc266c8c0 >>> packet_type 0x691 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 >>> rss 0xde0c21da fdir.hi 0x0 fdir.lo 0xde0c21da >>> Packet Offload Flags >>> PKT_RX_RSS_HASH (0x0002) RX packet with RSS hash result >>> PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid >>> PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid >>> Packet Types >>> RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet >>> RTE_PTYPE_L3_IPV4_EXT_UNKNOWN (0x0090) IPv4 packet with or without >>> extension headers >>> RTE_PTYPE_L4_NONFRAG (0x0600) Non-fragmented IP packet >>> IP4: fa:16:3e:4b:6b:42 -> fa:16:3e:97:04:19 802.1q vlan 1556 >>> IPSEC_ESP: 20.20.99.215 -> 80.80.80.80 >>> tos 0x00, ttl 64, length 120, checksum 0x21c9 dscp CS0 ecn NON_ECN >>> fragment id 0x0000, flags DONT_FRAGMENT >>> 00:03:26:875702: ethernet-input >>> frame: flags 0x3, hw-if-index 3, sw-if-index 3 >>> IP4: fa:16:3e:4b:6b:42 -> fa:16:3e:97:04:19 802.1q vlan 1556 >>> 00:03:26:875708: ip4-input >>> IPSEC_ESP: 20.20.99.215 -> 80.80.80.80 >>> tos 0x00, ttl 64, length 120, checksum 0x21c9 dscp CS0 ecn NON_ECN >>> fragment id 0x0000, flags DONT_FRAGMENT >>> 00:03:26:875714: ip4-lookup >>> fib 2 dpo-idx 27 flow hash: 0x00000000 >>> IPSEC_ESP: 20.20.99.215 -> 80.80.80.80 >>> tos 0x00, ttl 64, length 120, checksum 0x21c9 dscp CS0 ecn NON_ECN >>> fragment id 0x0000, flags DONT_FRAGMENT >>> 00:03:26:875715: ip4-local >>> IPSEC_ESP: 20.20.99.215 -> 80.80.80.80 >>> tos 0x00, ttl 64, length 120, checksum 0x21c9 dscp CS0 ecn NON_ECN >>> fragment id 0x0000, flags DONT_FRAGMENT >>> 00:03:26:875717: ipsec4-tun-input >>> IPSec: remote:20.20.99.215 spi:4213 (0x00001075) sa:1 tun:0 seq 1 sa >>> -2029505104 >>> 00:03:26:875719: esp4-decrypt-tun >>> esp: crypto aes-cbc-256 integrity sha1-96 pkt-seq 1 sa-seq 1 sa-seq-hi 0 >>> 00:03:26:875801: ip4-input-no-checksum >>> TCP: 44.44.44.44 -> 30.30.30.30 >>> tos 0x00, ttl 64, length 60, checksum 0x3cfa dscp CS0 ecn NON_ECN >>> fragment id 0x692e, flags DONT_FRAGMENT >>> TCP: 45723 -> 1234 >>> seq. 0x119d35d0 ack 0x00000000 >>> flags 0x02 SYN, tcp header: 40 bytes >>> window 64240, checksum 0xd6be >>> 00:03:26:875803: ip4-lookup >>> fib 2 dpo-idx 25 flow hash: 0x00000000 >>> TCP: 44.44.44.44 -> 30.30.30.30 >>> tos 0x00, ttl 64, length 60, checksum 0x3cfa dscp CS0 ecn NON_ECN >>> fragment id 0x692e, flags DONT_FRAGMENT >>> TCP: 45723 -> 1234 >>> seq. 0x119d35d0 ack 0x00000000 >>> flags 0x02 SYN, tcp header: 40 bytes >>> window 64240, checksum 0xd6be >>> 00:03:26:875803: ip4-local >>> TCP: 44.44.44.44 -> 30.30.30.30 >>> tos 0x00, ttl 64, length 60, checksum 0x3cfa dscp CS0 ecn NON_ECN >>> fragment id 0x692e, flags DONT_FRAGMENT >>> TCP: 45723 -> 1234 >>> seq. 0x119d35d0 ack 0x00000000 >>> flags 0x02 SYN, tcp header: 40 bytes >>> window 64240, checksum 0xd6be >>> 00:03:26:875805: an_ppe_router_input >>> AN_PPE_ROUTER_INPUT: sw_if_index 1, next index 3 >>> >>> 00:03:26:875819: tcp4-input >>> [0:0][T] :::0->:::0 state CLOSED >>> TCP: 45723 -> 1234 >>> seq. 0x119d35d0 ack 0x00000000 >>> flags 0x02 SYN, tcp header: 40 bytes >>> window 64240, checksum 0xd6be >>> 00:03:26:875826: tcp4-listen >>> 1234 -> 45723 (LISTEN) >>> >>> >>> Interface details >>> ==================== >>> vpp# show interface >>> Name Idx State MTU (L3/IP4/IP6/MPLS) >>> Counter Count >>> VirtualFuncEthernet0/7/0 3 up 1500/0/0/0 rx >>> packets 3922 >>> rx >>> bytes 239610 >>> tx >>> packets 17 >>> tx >>> bytes 6704 >>> drops >>> 3914 >>> VirtualFuncEthernet0/7/0.1557 13 up 1500/0/0/0 >>> VirtualFuncEthernet0/7/0.1556 14 up 1500/0/0/0 rx >>> packets 22 >>> rx >>> bytes 5130 >>> tx >>> packets 17 >>> tx >>> bytes 6704 >>> drops >>> 14 >>> ip4 >>> 19 >>> VirtualFuncEthernet0/8/0 4 down 9000/0/0/0 >>> gre1 2 up 9000/0/0/0 >>> ipip0 1 up 9000/0/0/0 rx >>> packets 5 >>> rx >>> bytes 500 >>> ip4 >>> 5 >>> local0 0 down 0/0/0/0 >>> loop2 8 up 9000/0/0/0 >>> loop3 9 up 9000/0/0/0 >>> loop4 10 up 9000/0/0/0 >>> loop5 11 up 9000/0/0/0 >>> loop6 12 up 9000/0/0/0 >>> memif0/0 5 up 0/65535/0/0 rx >>> packets 1036 >>> rx >>> bytes 67766832 >>> tx >>> packets 1049 >>> tx >>> bytes 67822842 >>> drops >>> 2 >>> ip4 >>> 1036 >>> memif128/0 6 up 0/65535/0/0 >>> memif192/0 7 up 0/65535/0/0 >>> memif210/0 15 up 0/65535/0/0 >>> memif210/1 16 up 0/65535/0/0 >>> vpp# >>> vpp# >>> >>> On Tue, Mar 15, 2022 at 2:32 PM Vijay Kumar via lists.fd.io >>> <http://lists.fd.io/> <vjkumar2003=gmail....@lists.fd.io >>> <mailto:gmail....@lists.fd.io>> wrote: >>> Hi experts, >>> >>> I recently integrated vpp stack 21.06 and am running my NAS TCP protocol >>> test cases. The pcap dispatch trace that I captured shows the TCP SYN >>> packets are coming to tcp-input() and then to tcp-listen() but from here >>> pkts are dropped instead of going to tcp-output() graph nodes for sending >>> the SYN-ACK >>> >>> NOTE >>> ========= >>> The session layer is enabled and the TCP listen IP/port is also showing in >>> the session verbose command. >>> >>> The same test case used to work fine in the VPP 20.05. Am I missing >>> anything? >>> Let me know if there is anything I need to look at. >>> >>> vpp# show session verbose >>> Connection State >>> Rx-f Tx-f >>> [0:0][T] 30.30.30.30:1234->0.0.0.0:0 <http://0.0.0.0:0/> >>> LISTEN 0 0 >>> Thread 0: active sessions 1 >>> Thread 1: no sessions >>> >>> >>> >>> >>> >>> >> >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#21033): https://lists.fd.io/g/vpp-dev/message/21033 Mute This Topic: https://lists.fd.io/mt/89793823/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
