Hi,
when using config
unix { interactive cli-listen /run/vpp/cli.sock gid 0 } cpu { main-core 1 }
dpdk { dev 0000:05:00.1 } *
* vpp# set int state TenGigabitEthernet5/0/1 up
vpp# set int ip addr TenGigabitEthernet5/0/1 192.168.1.1/24
vpp# ping 192.168.1.254 repeat 6 verbose
Source address: 192.168.1.1
64 bytes from 192.168.1.254: icmp_seq=3 ttl=64 time=.2041 ms *WORK AS EXPECTED
BUT:
* add corelist-workers 2,3 in config
unix { interactive cli-listen /run/vpp/cli.sock gid 0 } cpu { main-core 1
*corelist-workers 2,3* } dpdk { dev 0000:05:00.1 }
ping *cant work,and kernel error:*
3月 08 11:52:09 h7 kernel: DMAR: DRHD: handling fault status reg 102
3月 08 11:52:09 h7 kernel: DMAR: DMAR:[DMA Write] Request device [05:00.1] fault
addr 85b738000
DMAR:[fault reason 05] PTE Write access is not set
3月 08 11:52:10 h7 kernel: DMAR: DRHD: handling fault status reg 202
3月 08 11:52:10 h7 kernel: DMAR: DMAR:[DMA Write] Request device [05:00.1] fault
addr 85b737000
DMAR:[fault reason 05] PTE Write access is not set
3月 08 11:52:11 h7 kernel: DMAR: DRHD: handling fault status reg 302
3月 08 11:52:11 h7 kernel: DMAR: DMAR:[DMA Write] Request device [05:00.1] fault
addr 85b736000
DMAR:[fault reason 05] PTE Write access is not set
3月 08 11:52:12 h7 kernel: DMAR: DRHD: handling fault status reg 402
3月 08 11:52:12 h7 kernel: DMAR: DMAR:[DMA Write] Request device [05:00.1] fault
addr 85b736000
DMAR:[fault reason 05] PTE Write access is not set
3月 08 11:52:13 h7 kernel: DMAR: DRHD: handling fault status reg 502
3月 08 11:52:13 h7 kernel: DMAR: DMAR:[DMA Write] Request device [05:00.1] fault
addr 85b735000
DMAR:[fault reason 05] PTE Write access is not set
3月 08 11:52:14 h7 kernel: DMAR: DRHD: handling fault status reg 602
3月 08 11:52:14 h7 kernel: DMAR: DMAR:[DMA Write] Request device [05:00.1] fault
addr 85b735000
DMAR:[fault reason 05] PTE Write access is not set
enviroment:
DELL R430 with 2 cpus
centos 7.4 kernel 3.10.0-693.17.1.el7.x86_64
IOMMU kernel parameter: intel_iommu=on iommu=pt
*FULL LOG:*
vpp# set int state TenGigabitEthernet5/0/1 up
vpp# set int ip addr TenGigabitEthernet5/0/1 192.168.1.1/24
vpp# trace add dpdk-input 100
vpp# clear hardware
vpp# clear interfaces
vpp# clear error
vpp# clear run
vpp# ping 192.168.1.254 repeat 6 verbose
Source address: 192.168.1.1 *( DMAR kernel error occur )*
Source address: 192.168.1.1
Source address: 192.168.1.1
Source address: 192.168.1.1
Source address: 192.168.1.1
Source address: 192.168.1.1
Statistics: 6 sent, 0 received, 100% packet loss
vpp# show trace
------------------- Start of thread 0 vpp_main -------------------
No packets in trace buffer
------------------- Start of thread 1 vpp_wk_0 -------------------
Packet 1
01:45:58:568674: dpdk-input
TenGigabitEthernet5/0/1 rx queue 0
buffer 0xa4c2f: current data 0, length 60, free-list 0, clone-count 0,
totlen-nifb 0, trace 0x0
ext-hdr-valid
l4-cksum-computed l4-cksum-correct l2-hdr-offset 0
PKT MBUF: port 0, nb_segs 1, pkt_len 60
buf_len 2176, data_len 60, ol_flags 0x180, data_off 128, phys_addr
0x5b730c40
packet_type 0x1 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
Packet Offload Flags
PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid
PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid
Packet Types
RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet
0x0000: 00:00:00:00:00:00 -> 00:00:00:00:00:00
*(like ALL packet data is 0)*
01:45:58:568718: ethernet-input
0x0000: 00:00:00:00:00:00 -> 00:00:00:00:00:00
01:45:58:568723: error-drop
ethernet-input: l3 mac mismatch
Packet 2
01:45:59:568713: dpdk-input
TenGigabitEthernet5/0/1 rx queue 0
buffer 0xa4c08: current data 0, length 60, free-list 0, clone-count 0,
totlen-nifb 0, trace 0x1
ext-hdr-valid
l4-cksum-computed l4-cksum-correct l2-hdr-offset 0
PKT MBUF: port 0, nb_segs 1, pkt_len 60
buf_len 2176, data_len 60, ol_flags 0x180, data_off 128, phys_addr
0x5b730280
packet_type 0x1 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
Packet Offload Flags
PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid
PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid
Packet Types
RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet
0x0000: 00:00:00:00:00:00 -> 00:00:00:00:00:00
01:45:59:568714: ethernet-input
0x0000: 00:00:00:00:00:00 -> 00:00:00:00:00:00
01:45:59:568716: error-drop
ethernet-input: l3 mac mismatch
Packet 3
01:46:00:573520: dpdk-input
TenGigabitEthernet5/0/1 rx queue 0
buffer 0xa4be1: current data 0, length 60, free-list 0, clone-count 0,
totlen-nifb 0, trace 0x2
ext-hdr-valid
l4-cksum-computed l4-cksum-correct l2-hdr-offset 0
PKT MBUF: port 0, nb_segs 1, pkt_len 60
buf_len 2176, data_len 60, ol_flags 0x180, data_off 128, phys_addr
0x5b72f8c0
packet_type 0x1 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
Packet Offload Flags
PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid
PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid
Packet Types
RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet
0x0000: 00:00:00:00:00:00 -> 00:00:00:00:00:00
01:46:00:573524: ethernet-input
0x0000: 00:00:00:00:00:00 -> 00:00:00:00:00:00
01:46:00:573525: error-drop
ethernet-input: l3 mac mismatch
Packet 4
01:46:01:578244: dpdk-input
TenGigabitEthernet5/0/1 rx queue 0
buffer 0xa4bba: current data 0, length 60, free-list 0, clone-count 0,
totlen-nifb 0, trace 0x3
ext-hdr-valid
l4-cksum-computed l4-cksum-correct l2-hdr-offset 0
PKT MBUF: port 0, nb_segs 1, pkt_len 60
buf_len 2176, data_len 60, ol_flags 0x180, data_off 128, phys_addr
0x5b72ef00
packet_type 0x1 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
Packet Offload Flags
PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid
PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid
Packet Types
RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet
0x0000: 00:00:00:00:00:00 -> 00:00:00:00:00:00
01:46:01:578269: ethernet-input
0x0000: 00:00:00:00:00:00 -> 00:00:00:00:00:00
01:46:01:578277: error-drop
ethernet-input: l3 mac mismatch
Packet 5
01:46:02:578194: dpdk-input
TenGigabitEthernet5/0/1 rx queue 0
buffer 0xa4b93: current data 0, length 60, free-list 0, clone-count 0,
totlen-nifb 0, trace 0x4
ext-hdr-valid
l4-cksum-computed l4-cksum-correct l2-hdr-offset 0
PKT MBUF: port 0, nb_segs 1, pkt_len 60
buf_len 2176, data_len 60, ol_flags 0x180, data_off 128, phys_addr
0x5b72e540
packet_type 0x1 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
Packet Offload Flags
PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid
PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid
Packet Types
RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet
0x0000: 00:00:00:00:00:00 -> 00:00:00:00:00:00
01:46:02:578199: ethernet-input
0x0000: 00:00:00:00:00:00 -> 00:00:00:00:00:00
01:46:02:578210: error-drop
ethernet-input: l3 mac mismatch
Packet 6
01:46:03:578592: dpdk-input
TenGigabitEthernet5/0/1 rx queue 0
buffer 0xa4b6c: current data 0, length 60, free-list 0, clone-count 0,
totlen-nifb 0, trace 0x5
ext-hdr-valid
l4-cksum-computed l4-cksum-correct l2-hdr-offset 0
PKT MBUF: port 0, nb_segs 1, pkt_len 60
buf_len 2176, data_len 60, ol_flags 0x180, data_off 128, phys_addr
0x5b72db80
packet_type 0x1 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
Packet Offload Flags
PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid
PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid
Packet Types
RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet
0x0000: 00:00:00:00:00:00 -> 00:00:00:00:00:00
01:46:03:578592: ethernet-input
0x0000: 00:00:00:00:00:00 -> 00:00:00:00:00:00
01:46:03:578594: error-drop
ethernet-input: l3 mac mismatch
------------------- Start of thread 2 vpp_wk_1 -------------------
No packets in trace buffer
vpp#
vpp# show hard
Name Idx Link Hardware
TenGigabitEthernet5/0/1 1 up TenGigabitEthernet5/0/1
Ethernet address a0:36:9f:9d:1e:62
Intel 82599
carrier up full duplex speed 10000 mtu 9216
rx queues 1, rx desc 1024, tx queues 3, tx desc 1024
cpu socket 0
tx frames ok 6
tx bytes ok 360
rx frames ok 6
rx bytes ok 360
extended stats:
rx good packets 6
tx good packets 6
rx good bytes 360
tx good bytes 360
rx q0packets 6
rx q0bytes 360
tx q0packets 6
tx q0bytes 252
rx size 64 packets 6
rx total packets 6
rx total bytes 360
tx total packets 6
tx size 64 packets 6
tx broadcast packets 6
out pkts untagged 6
local0 0 down local0
local
vpp# show interface
Name Idx State Counter
Count
TenGigabitEthernet5/0/1 1 up rx packets
6
rx bytes
360
tx packets
6
tx bytes
252
drops
12
local0 0 down
vpp# show run
Thread 0 vpp_main (lcore 1)
Time 106.2, average vectors/node 1.00, last 128 main loops 0.00 per node 0.00
vector rates in 0.0000e0, out 5.6503e-2, drop 5.6503e-2, punt 0.0000e0
Name State Calls Vectors
Suspends Clocks Vectors/Call
TenGigabitEthernet5/0/1-output active 6 6
0 3.87e3 1.00
TenGigabitEthernet5/0/1-tx active 6 6
0 3.10e3 1.00
api-rx-from-ring any wait 0 0
5 6.49e3 0.00
cdp-process any wait 0 0
11 4.41e3 0.00
dhcp-client-process any wait 0 0
1 1.57e4 0.00
dpdk-process any wait 0 0
35 1.98e5 0.00
error-drop active 6 6
0 6.78e3 1.00
fib-walk any wait 0 0
53 2.15e3 0.00
ikev2-manager-process any wait 0 0
106 1.35e3 0.00
ip-route-resolver-process any wait 0 0
1 8.95e3 0.00
ip4-glean active 6 6
0 5.96e3 1.00
ip4-lookup active 6 6
0 7.34e3 1.00
ip4-reassembly-expire-walk any wait 0 0
11 3.03e3 0.00
ip6-icmp-neighbor-discovery-ev any wait 0 0
106 1.33e3 0.00
ip6-reassembly-expire-walk any wait 0 0
11 2.82e3 0.00
lisp-retry-service any wait 0 0
53 2.54e3 0.00
unix-cli-stdin active 0 0
83 4.77e5 0.00
unix-epoll-input polling 1018713 0
0 2.76e5 0.00
vpe-oam-process any wait 0 0
52 1.47e3 0.00
---------------
Thread 1 vpp_wk_0 (lcore 2)
Time 106.2, average vectors/node 1.00, last 128 main loops 0.00 per node 0.00
vector rates in 5.6503e-2, out 0.0000e0, drop 5.6503e-2, punt 0.0000e0
Name State Calls Vectors
Suspends Clocks Vectors/Call
dpdk-input polling 1832771082 6
0 2.83e10 0.00
error-drop active 6 6
0 4.41e3 1.00
ethernet-input active 6 6
0 1.17e4 1.00
---------------
Thread 2 vpp_wk_1 (lcore 3)
Time 106.2, average vectors/node 0.00, last 128 main loops 0.00 per node 0.00
vector rates in 0.0000e0, out 0.0000e0, drop 0.0000e0, punt 0.0000e0
Name State Calls Vectors
Suspends Clocks Vectors/Call
unix-epoll-input polling 809061 0
0 2.77e5 0.00
vpp#
vpp#
vpp# show error
Count Node Reason
6 ip4-glean ARP requests sent
6 ethernet-input l3 mac mismatch
vpp#
vpp#
vpp# show version verbose
Version: v18.04-rc0~361-ga36f08c~b3845
Compiled by: root
Compile host: a9d16d538da8
Compile date: Wed Mar 7 23:47:04 UTC 2018
Compile location: /w/workspace/vpp-merge-master-centos7
Compiler: GCC 4.8.5 20150623 (Red Hat 4.8.5-16)
Current PID: 28616
vpp#
vpp#
vpp# show threads
ID Name Type LWP Sched Policy (Priority) lcore
Core Socket State
0 vpp_main 28616 other (0) 1
0 1
1 vpp_wk_0 workers 28620 other (0) 2
1 0
2 vpp_wk_1 workers 28621 other (0) 3
1 1
3 stats 28622 other (0) 0
0 0
vpp#
vpp#
vpp# show cpu
Model name: Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz
Microarchitecture: Haswell (Haswell E)
Flags: sse3 ssse3 sse41 sse42 avx avx2 aes invariant_tsc
Base frequency: 2.40 GHz
vpp#
vpp#
vpp# show pci
Address Sock VID:PID Link Speed Driver Product Name
Vital Product Data
0000:02:00.0 0 14e4:165f 5.0 GT/s x1 tg3 Broadcom NetXtreme
Gigabit Ether
0000:02:00.1 0 14e4:165f 5.0 GT/s x1 tg3 Broadcom NetXtreme
Gigabit Ether
0000:03:00.0 0 14e4:165f 5.0 GT/s x1 tg3 Broadcom NetXtreme
Gigabit Ether
0000:03:00.1 0 14e4:165f 5.0 GT/s x1 tg3 Broadcom NetXtreme
Gigabit Ether
0000:05:00.0 0 8086:154d 5.0 GT/s x8 ixgbe X520 10GbE
Controller
0000:05:00.1 0 8086:154d 5.0 GT/s x8 vfio-pci X520 10GbE
Controller
0000:05:10.0 0 8086:10ed unknown ixgbevf
0000:05:10.2 0 8086:10ed unknown ixgbevf
0000:05:10.4 0 8086:10ed unknown ixgbevf
0000:05:10.6 0 8086:10ed unknown ixgbevf
0000:05:11.0 0 8086:10ed unknown ixgbevf
0000:05:11.2 0 8086:10ed unknown ixgbevf
0000:05:11.4 0 8086:10ed unknown ixgbevf
0000:05:11.6 0 8086:10ed unknown ixgbevf
vpp#
vpp#
*-----LOG END--------
*
version tested and work fine: (all fetched from nexus.fd.io)
18.01.1-4~geed6a2b~b56
18.04-rc0~287_gf9a968e~b3764
18.04-rc0~323_g7b18536~b3800
18.04-rc0~325_g2060db8~b3801
18.04-rc0~326_g599a16b~b3803
18.04-rc0~327_g03f942a~b3804
18.04-rc0~328_ge3e2f07~b3805
18.04-rc0~329_gd5ded2d~b3806
version tested and cant work: (all fetched from nexus.fd.io)
18.04-rc0~361_ga36f08c~b3845
18.04-rc0~333_gf313b74~b3810
18.04-rc0~331_geac3b11~b3808
git log --pretty=oneline --abbrev-commit
...
9e09ff39 SCTP: retransmission in INIT/SHUTDOWN phase
eac3b11b vlib: vlib_buffer_free_internal optimizations
*(~331)*
1ba0fa4b vlib: vfio code rework
d5ded2df vlib: rework PCI INTx support
*(~329)*
e3e2f071 tls: add stop listen handler
03f942a1 NAT44 - unknown protocols work with forwarding
...
caused by 1ba0fa4b or eac3b11b ?
thanks,
Wuxp
