This looks like a DPDK issue, but I’ll let Damjan be the judge of that. To see if this is a config issues, could you simplify your startup config by - removing “workers 0” from the two nics and adding “num-rx-queues 2” to the nics or to the default stanza, if you’re running with 2 workers - comment out the cryptodev config
If the two nics don’t come up, check if there’s any obvious dpdk error in “show log”. Florin > On Oct 17, 2019, at 4:56 PM, Chuan Han via Lists.Fd.Io > <chuanhan=google....@lists.fd.io> wrote: > > I tried disabling autoneg on R740 side. It is not allowed too. If vpp cannot > allow two nics to be successfully added to the same vpp instance, it seems to > be a bug. Is it something which can be easily spotted in the code base? > > It is also not possible to enforce symmetricity on internet. The other party > can do anything as long as basic ping works. > > On Thu, Oct 17, 2019 at 3:55 PM Chuan Han <chuan...@google.com > <mailto:chuan...@google.com>> wrote: > If I only put one phy nic, i.e., eth0, to vpp, 'sh hardware' shows it is up. > If I put both eth0 and eth1 in vpp, eth0 is always down. It seems something > is wrong with the nic or vpp does not support this type of hardware? > > We tried enabling autoneg on R230. It is not allowed. To avoid asymmetric > settings, disabling autoneg on R740 will help? > > On Thu, Oct 17, 2019 at 3:46 PM Balaji Venkatraman (balajiv) > <bala...@cisco.com <mailto:bala...@cisco.com>> wrote: > It plays a role if it is asymmetric at both ends. You could enable it at both > ends and check. > >> On Oct 17, 2019, at 3:15 PM, Chuan Han <chuan...@google.com >> <mailto:chuan...@google.com>> wrote: >> >> >> I rebooted the r230 machine and found the phy nic corresponding to eth has >> autoneg off. >> >> root@esdn-relay:~/gnxi/perf_testing/r230# ethtool enp6s0f1 >> Settings for enp6s0f1: >> Supported ports: [ FIBRE ] >> Supported link modes: 10000baseT/Full >> Supported pause frame use: Symmetric >> Supports auto-negotiation: No >> Supported FEC modes: Not reported >> Advertised link modes: 10000baseT/Full >> Advertised pause frame use: Symmetric >> Advertised auto-negotiation: No >> Advertised FEC modes: Not reported >> Speed: 10000Mb/s >> Duplex: Full >> Port: Direct Attach Copper >> PHYAD: 0 >> Transceiver: internal >> Auto-negotiation: off >> Supports Wake-on: d >> Wake-on: d >> Current message level: 0x00000007 (7) >> drv probe link >> Link detected: yes >> root@esdn-relay:~/gnxi/perf_testing/r230# >> >> On r740, autoneg is on. It is copper. >> >> root@esdn-lab:~/gnxi/perf_testing/r740/vpp# ethtool eno3 >> Settings for eno3: >> Supported ports: [ TP ] >> Supported link modes: 100baseT/Full >> 1000baseT/Full >> 10000baseT/Full >> Supported pause frame use: Symmetric >> Supports auto-negotiation: Yes >> Supported FEC modes: Not reported >> Advertised link modes: 100baseT/Full >> 1000baseT/Full >> 10000baseT/Full >> Advertised pause frame use: Symmetric >> Advertised auto-negotiation: Yes >> Advertised FEC modes: Not reported >> Speed: 10000Mb/s >> Duplex: Full >> Port: Twisted Pair >> PHYAD: 0 >> Transceiver: internal >> Auto-negotiation: on >> MDI-X: Unknown >> Supports Wake-on: umbg >> Wake-on: g >> Current message level: 0x00000007 (7) >> drv probe link >> Link detected: yes >> root@esdn-lab:~/gnxi/perf_testing/r740/vpp# >> >> not clear if this plays a role or not. >> >> On Thu, Oct 17, 2019 at 2:41 PM Chuan Han via Lists.Fd.Io >> <http://lists.fd.io/> <chuanhan=google....@lists.fd.io >> <mailto:google....@lists.fd.io>> wrote: >> Restarting ixia controller does not help. We ended up with both ixia ports >> having '!'. >> >> We are not sure how ixia port plays a role here. eth0 interfaces are the >> interfaces connecting two servers, not to ixia. >> >> On Thu, Oct 17, 2019 at 11:26 AM Balaji Venkatraman (balajiv) >> <bala...@cisco.com <mailto:bala...@cisco.com>> wrote: >> Hi Chuan, >> >> >> >> Could you please try to reset the ixia controller connected to port 4? >> >> I have seen issues with ‘!’ on ixia. Given the carrier on eth0 is down, I >> suspect the ixia port. >> >> >> >> -- >> >> Regards, >> >> Balaji. >> >> >> >> >> >> From: Chuan Han <chuan...@google.com <mailto:chuan...@google.com>> >> Date: Thursday, October 17, 2019 at 11:09 AM >> To: "Balaji Venkatraman (balajiv)" <bala...@cisco.com >> <mailto:bala...@cisco.com>> >> Cc: "vpp-dev@lists.fd.io <mailto:vpp-dev@lists.fd.io>" <vpp-dev@lists.fd.io >> <mailto:vpp-dev@lists.fd.io>>, Arivudainambi Appachi gounder >> <aappa...@google.com <mailto:aappa...@google.com>>, Jerry Cen >> <zhiw...@google.com <mailto:zhiw...@google.com>> >> Subject: Re: [vpp-dev] Basic l2 bridging does not work >> >> >> >> Yes. It is unidirectional stream from port 1 to port 4. >> >> >> >> Another engineer, Nambi, configured ixia. What he showed me yesterday is >> that xia port connected to port 1 is green and good. ixia port connected to >> port 4 is green but has a red exclamation mark, which means ping does not >> work. >> >> >> >> We also found eth0 on R230 is down shown by "show hardware eth0" command. >> However "show int" shows it is up. >> >> >> >> >> >> vpp# sh hardware-interfaces eth0 >> Name Idx Link Hardware >> eth0 2 down eth0 >> Link speed: unknown >> Ethernet address b4:96:91:23:1e:d6 >> Intel 82599 >> carrier down >> flags: admin-up promisc pmd rx-ip4-cksum >> rx: queues 1 (max 128), desc 512 (min 32 max 4096 align 8) >> tx: queues 3 (max 64), desc 512 (min 32 max 4096 align 8) >> pci: device 8086:154d subsystem 8086:7b11 address 0000:06:00.01 numa 0 >> max rx packet len: 15872 >> promiscuous: unicast on all-multicast on >> vlan offload: strip off filter off qinq off >> rx offload avail: vlan-strip ipv4-cksum udp-cksum tcp-cksum tcp-lro >> macsec-strip vlan-filter vlan-extend jumbo-frame >> scatter >> security keep-crc >> rx offload active: ipv4-cksum >> tx offload avail: vlan-insert ipv4-cksum udp-cksum tcp-cksum sctp-cksum >> tcp-tso macsec-insert multi-segs security >> tx offload active: none >> rss avail: ipv4-tcp ipv4-udp ipv4 ipv6-tcp-ex ipv6-udp-ex >> ipv6-tcp >> ipv6-udp ipv6-ex ipv6 >> rss active: none >> tx burst function: (nil) >> rx burst function: ixgbe_recv_pkts_vec >> >> rx frames ok 33278 >> rx bytes ok 3960082 >> extended stats: >> rx good packets 33278 >> rx good bytes 3960082 >> rx q0packets 33278 >> rx q0bytes 3960082 >> rx size 65 to 127 packets 33278 >> rx multicast packets 33278 >> rx total packets 33278 >> rx total bytes 3960082 >> vpp# sh int >> Name Idx State MTU (L3/IP4/IP6/MPLS) >> Counter Count >> eth0 2 up 9000/0/0/0 rx >> packets 33279 >> rx bytes >> 3960201 >> drops >> 5 >> punt >> 1 >> tx-error >> 33274 >> eth1 1 up 9000/0/0/0 rx >> packets 33274 >> rx bytes >> 3959606 >> tx >> packets 33273 >> tx bytes >> 3959487 >> drops >> 33274 >> tx-error >> 3 >> local0 0 down 0/0/0/0 >> vpp# >> >> >> >> On Thu, Oct 17, 2019 at 10:54 AM Balaji Venkatraman (balajiv) >> <bala...@cisco.com <mailto:bala...@cisco.com>> wrote: >> >> Hi Chuan, >> >> >> >> I assume u have unidirectional stream ? ixia->1->2->3->4->ixia? >> >> >> >> vpp# sh int >> Name Idx State MTU (L3/IP4/IP6/MPLS) >> Counter Count >> eth0 2 up 9000/0/0/0 rx >> packets 30925 >> rx bytes >> 3680075 >> drops >> 5 >> punt >> 1 >> tx-error >> 30920 >> eth1 1 up 9000/0/0/0 rx >> packets 30920 <<< packets are received on port 3 >> rx bytes >> 3679480 >> tx >> packets 30919 >> tx bytes >> 3679361 >> drops >> 30920 <<< all dropped at port 3 >> tx-error >> 3 >> local0 0 down 0/0/0/0 >> >> >> >> On sh error logs on R 230 we see >> >> 1 ethernet-input l3 mac mismatch <<<< >> 3 eth1-output interface is down >> 30922 eth0-output interface is down >> >> >> >> >> >> Do u see the arp getting resolved on ixia? The mac on ixia at port with >> 172.16.1.2/24 <http://172.16.1.2/24> should be seen on its other port. Are >> the ixia ports up at both ends? >> >> >> >> >> >> -- >> >> Regards, >> >> Balaji. >> >> >> >> >> >> From: <vpp-dev@lists.fd.io <mailto:vpp-dev@lists.fd.io>> on behalf of "Chuan >> Han via Lists.Fd.Io <http://lists.fd.io/>" <chuanhan=google....@lists.fd.io >> <mailto:google....@lists.fd.io>> >> Reply-To: "chuan...@google.com <mailto:chuan...@google.com>" >> <chuan...@google.com <mailto:chuan...@google.com>> >> Date: Thursday, October 17, 2019 at 9:59 AM >> To: "Balaji Venkatraman (balajiv)" <bala...@cisco.com >> <mailto:bala...@cisco.com>> >> Cc: "vpp-dev@lists.fd.io <mailto:vpp-dev@lists.fd.io>" <vpp-dev@lists.fd.io >> <mailto:vpp-dev@lists.fd.io>> >> Subject: Re: [vpp-dev] Basic l2 bridging does not work >> >> >> >> It seems R740 vpp works fine. All packets coming from port 1 go to port 2. >> >> >> >> vpp# sh int >> Name Idx State MTU (L3/IP4/IP6/MPLS) >> Counter Count >> eth0 2 up 9000/0/0/0 tx >> packets 30895 >> tx bytes >> 3676505 >> eth1 1 up 9000/0/0/0 rx >> packets 30895 >> rx bytes >> 3676505 >> local0 0 down 0/0/0/0 >> vpp# sh int >> Name Idx State MTU (L3/IP4/IP6/MPLS) >> Counter Count >> eth0 2 up 9000/0/0/0 tx >> packets 30897 >> tx bytes >> 3676743 >> eth1 1 up 9000/0/0/0 rx >> packets 30897 >> rx bytes >> 3676743 >> local0 0 down 0/0/0/0 >> vpp# sh error >> Count Node Reason >> 30899 l2-output L2 output packets >> 30899 l2-learn L2 learn packets >> 1 l2-learn L2 learn misses >> 30899 l2-input L2 input packets >> 30899 l2-flood L2 flood packets >> vpp# >> >> >> >> The drop happened on R230 vpp. Port 3 dropped all pkts complaining about >> down interface. However, show command shows interfaces are up. >> >> >> >> vpp# sh int >> Name Idx State MTU (L3/IP4/IP6/MPLS) >> Counter Count >> eth0 2 up 9000/0/0/0 rx >> packets 30925 >> rx bytes >> 3680075 >> drops >> 5 >> punt >> 1 >> tx-error >> 30920 >> eth1 1 up 9000/0/0/0 rx >> packets 30920 >> rx bytes >> 3679480 >> tx >> packets 30919 >> tx bytes >> 3679361 >> drops >> 30920 >> tx-error >> 3 >> local0 0 down 0/0/0/0 >> vpp# sh error >> Count Node Reason >> 2 llc-input unknown llc ssap/dsap >> 61846 l2-output L2 output packets >> 61846 l2-learn L2 learn packets >> 2 l2-learn L2 learn misses >> 61846 l2-input L2 input packets >> 61846 l2-flood L2 flood packets >> 1 ethernet-input l3 mac mismatch >> 3 eth1-output interface is down >> 30922 eth0-output interface is down >> vpp# >> >> >> >> Not sure how to check mac issues. Can you explain a bit more? Here is what I >> can see on R230 vpp. >> >> >> >> vpp# show bridge-domain 1 detail >> BD-ID Index BSN Age(min) Learning U-Forwrd UU-Flood Flooding >> ARP-Term arp-ufwd BVI-Intf >> 1 1 0 off on on flood on >> off off N/A >> >> Interface If-idx ISN SHG BVI TxFlood >> VLAN-Tag-Rewrite >> eth0 2 1 0 - * >> none >> eth1 1 1 0 - * >> none >> vpp# sh l2fib verbose >> Mac-Address BD-Idx If-Idx BSN-ISN Age(min) static filter bvi >> Interface-Name >> 28:99:3a:f4:3a:a6 1 2 0/1 - - - - >> eth0 >> 28:99:3a:f4:3a:9c 1 1 0/1 - - - - >> eth1 >> L2FIB total/learned entries: 2/2 Last scan time: 0.0000e0sec Learn limit: >> 4194304 >> vpp# >> >> >> >> On Wed, Oct 16, 2019 at 6:01 PM Balaji Venkatraman (balajiv) >> <bala...@cisco.com <mailto:bala...@cisco.com>> wrote: >> >> +-------------------------------------------------------------------------+ >> >> | | >> >> | | >> >> | IXIA | >> >> | | >> >> | | >> >> +-----------------------------------------------------------^-------------+ >> >> |172.16.1.1/24 <http://172.16.1.1/24> >> |172.16.1.2/24 <http://172.16.1.2/24> >> | | >> >> | | >> >> |eth0 | eth0 >> >> +-----------v-------------+ +------------+-----------+ >> >> | 1 | | 4 | >> >> | | | | >> >> | | | | >> >> | | | | >> >> | |eth1 eth1 | | >> >> | VPP1 2 +--------------------> 3 VPP 2 | >> >> | | | | >> >> | | | | >> >> | | | | >> >> | | | | >> >> +-------------------------+ +------------------------+ >> >> R 740 R 230 >> >> >> >> >> >> Might help if you could check if the packet counts at ingress (port 1) & >> egress (port 2) match. Similarly 3 & 4. And the mac entries seen on both >> vpp(s). ARP req/rep tracing might also help. >> >> >> >> >> >> /- >> >> Balaji >> >> >> >> >> >> >> >> >> >> From: <vpp-dev@lists.fd.io <mailto:vpp-dev@lists.fd.io>> on behalf of >> "Damjan Marion via Lists.Fd.Io <http://lists.fd.io/>" >> <dmarion=me....@lists.fd.io <mailto:me....@lists.fd.io>> >> Reply-To: "dmar...@me.com <mailto:dmar...@me.com>" <dmar...@me.com >> <mailto:dmar...@me.com>> >> Date: Wednesday, October 16, 2019 at 5:12 PM >> To: "chuan...@google.com <mailto:chuan...@google.com>" <chuan...@google.com >> <mailto:chuan...@google.com>> >> Cc: "vpp-dev@lists.fd.io <mailto:vpp-dev@lists.fd.io>" <vpp-dev@lists.fd.io >> <mailto:vpp-dev@lists.fd.io>> >> Subject: Re: [vpp-dev] Basic l2 bridging does not work >> >> >> >> >> >> On 16 Oct 2019, at 16:14, Chuan Han via Lists.Fd.Io <http://lists.fd.io/> >> <chuanhan=google....@lists.fd.io <mailto:chuanhan=google....@lists.fd.io>> >> wrote: >> >> >> >> Hi, vpp experts, >> >> >> >> We are trying to make basic l2 bridge works within vpp. >> >> >> >> We have two servers: r230 and r740, each of which has two phy nics. Two >> servers are connected via cable. On each server, we bring these two nics >> into the same vpp instance and put them into the same l2 bridge. We tried >> sending traffic using ixia. However, ixia shows ping does not work. >> >> >> >> I attached the topology, vpp conf files, startup conf file, and logs. >> >> >> >> Please advise where we could make it wrong. >> >> >> >> Thanks. >> >> Chuan >> >> <r230 vpp.conf><r740 vpp.conf><r230 vpp startup.cfg><r740 vpp >> startup.cfg><r740.log><r230.log><vpp testbed - >> bridge.pdf>-=-=-=-=-=-=-=-=-=-=-=- >> Links: You receive all messages sent to this group. >> >> View/Reply Online (#14189): https://lists.fd.io/g/vpp-dev/message/14189 >> <https://lists.fd.io/g/vpp-dev/message/14189> >> Mute This Topic: https://lists.fd.io/mt/34655826/675642 >> <https://lists.fd.io/mt/34655826/675642> >> Group Owner: vpp-dev+ow...@lists.fd.io <mailto:vpp-dev+ow...@lists.fd.io> >> Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub >> <https://lists.fd.io/g/vpp-dev/unsub> [dmar...@me.com >> <mailto:dmar...@me.com>] >> -=-=-=-=-=-=-=-=-=-=-=- >> >> >> >> On the 1st look everything look ok including the packet trace. >> >> Can you try to clear counters* and enable packet trace on both instances. >> >> Then send known number of packets and find put where drop happens by looking >> into same outputs you already shared..... >> >> >> >> * "clear int", "clear run", "clear trace" >> >> >> >> -- >> Damjan >> >> >> >> -=-=-=-=-=-=-=-=-=-=-=- >> Links: You receive all messages sent to this group. >> >> View/Reply Online (#14211): https://lists.fd.io/g/vpp-dev/message/14211 >> <https://lists.fd.io/g/vpp-dev/message/14211> >> Mute This Topic: https://lists.fd.io/mt/34655826/1991531 >> <https://lists.fd.io/mt/34655826/1991531> >> Group Owner: vpp-dev+ow...@lists.fd.io <mailto:vpp-dev%2bow...@lists.fd.io> >> Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub >> <https://lists.fd.io/g/vpp-dev/unsub> [chuan...@google.com >> <mailto:chuan...@google.com>] >> -=-=-=-=-=-=-=-=-=-=-=- > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > > View/Reply Online (#14219): https://lists.fd.io/g/vpp-dev/message/14219 > <https://lists.fd.io/g/vpp-dev/message/14219> > Mute This Topic: https://lists.fd.io/mt/34655826/675152 > <https://lists.fd.io/mt/34655826/675152> > Group Owner: vpp-dev+ow...@lists.fd.io <mailto:vpp-dev+ow...@lists.fd.io> > Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub > <https://lists.fd.io/g/vpp-dev/unsub> [fcoras.li...@gmail.com > <mailto:fcoras.li...@gmail.com>] > -=-=-=-=-=-=-=-=-=-=-=-
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#14220): https://lists.fd.io/g/vpp-dev/message/14220 Mute This Topic: https://lists.fd.io/mt/34655826/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
