So tell us: what is the interface in question? How was it created?
Was vec_len(hi->hw_address) 6 when the interface was created? At the time of
the crash, was the hardwre address hw->hw_address intact but with an incorrect
length, or was it trash?
Haven’t seen any previous report of such misbehavior, which makes me wonder if
the issue is peculiar to your code, use-case, interface type / handling, etc.
FWIW... Dave
From: vpp-dev@lists.fd.io <vpp-dev@lists.fd.io> On Behalf Of Rajith PR via
lists.fd.io
Sent: Sunday, June 28, 2020 12:27 PM
To: vpp-dev <vpp-dev@lists.fd.io>
Subject: [vpp-dev] ASSERT in arp_mk_reply
Hi All,
We are seeing ASSERT (vec_len (hw_if0->hw_address) == 6); being hit in
arp_mk_reply() . This is happening on 19.08.
We are having worker threads and a main thread.
As such , the hw_if0 appears to be valid(the pointer and content).
But the length of the vector is 15.
I have attached some information from the core file.
Thread 11 (Thread 0x7f085f7fe700 (LWP 261)):
#0 vlib_get_trace_count (vm=0x7f08bc310480, rt=0x7f08b9a3d0c0) at
/development/libvpp/src/vlib/trace_funcs.h:177
#1 0x00007f092b98082c in rtb_vpp_shm_device_input (vm=0x7f08bc310480,
shmm=0x7f092bbe9980 <rtb_vpp_shm_main_ctx>, shmif=0x7f08bbdf93c0,
node=0x7f08b9a3d0c0,
frame=0x0, thread_index=2, queue_id=0) at
/development/libvpp/src/vpp/rtbrick/rtb_vpp_shm_node.c:341
#2 0x00007f092b98102e in rtb_vpp_shm_input_node_fn (vm=0x7f08bc310480,
node=0x7f08b9a3d0c0, f=0x0) at
/development/libvpp/src/vpp/rtbrick/rtb_vpp_shm_node.c:434
#3 0x00007f092a020c4f in dispatch_node (vm=0x7f08bc310480,
node=0x7f08b9a3d0c0, type=VLIB_NODE_TYPE_INPUT,
dispatch_state=VLIB_NODE_STATE_POLLING, frame=0x0,
last_time_stamp=3193893401554218) at
/development/libvpp/src/vlib/main.c:1207
#4 0x00007f092a022d9c in vlib_main_or_worker_loop (vm=0x7f08bc310480,
is_main=0) at /development/libvpp/src/vlib/main.c:1779
#5 0x00007f092a0238d1 in vlib_worker_loop (vm=0x7f08bc310480) at
/development/libvpp/src/vlib/main.c:1934
#6 0x00007f092a062306 in vlib_worker_thread_fn (arg=0x7f08b9749140) at
/development/libvpp/src/vlib/threads.c:1754
#7 0x00007f0929ce3600 in clib_calljmp () from
/usr/local/lib/libvppinfra.so.1.0.1
#8 0x00007f085f7fdec0 in ?? ()
#9 0x00007f092a05cb92 in vlib_worker_thread_bootstrap_fn (arg=0x7f08b9749140)
at /development/libvpp/src/vlib/threads.c:573
Backtrace stopped: previous frame inner to this frame (corrupt stack?)
Thread 10 (Thread 0x7f085ffff700 (LWP 260)):
#0 0x00007f0935cb86c2 in __GI___waitpid (pid=7605,
stat_loc=stat_loc@entry=0x7f08bc73db18, options=options@entry=0) at
../sysdeps/unix/sysv/linux/waitpid.c:30
#1 0x00007f0935c23067 in do_system (line=<optimized out>) at
../sysdeps/posix/system.c:149
#2 0x00007f093673c21a in bd_signal_handler_cb (signo=6) at
/development/librtbrickinfra/bd/src/bd.c:770
#3 0x00007f092a088d17 in rtb_bd_signal_handler (signo=6) at
/development/libvpp/src/vlib/unix/main.c:80
#4 0x00007f092a0890b2 in unix_signal_handler (signum=6, si=0x7f08bc73e2f0,
uc=0x7f08bc73e1c0) at /development/libvpp/src/vlib/unix/main.c:180
#5 <signal handler called>
#6 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#7 0x00007f0935c14801 in __GI_abort () at abort.c:79
#8 0x00007f092b827476 in os_panic () at
/development/libvpp/src/vpp/vnet/main.c:559
#9 0x00007f0929cc2825 in debugger () at
/development/libvpp/src/vppinfra/error.c:84
#10 0x00007f0929cc2bf4 in _clib_error (how_to_die=2, function_name=0x0,
line_number=0, fmt=0x7f092b1156b8 "%s:%d (%s) assertion `%s' fails")
at /development/libvpp/src/vppinfra/error.c:143
#11 0x00007f092aa096bf in arp_mk_reply (vnm=0x7f092b5a05a0 <vnet_main>,
p0=0x1001e349c0, sw_if_index0=8, if_addr0=0x7f08bbe399f4, arp0=0x1001e34b0e,
eth_rx=0x1001e34b00) at /development/libvpp/src/vnet/ethernet/arp.c:1206
#12 0x00007f092aa09e76 in arp_reply (vm=0x7f08bc30fd40, node=0x7f08bcc4dc80,
frame=0x7f08be1835c0) at /development/libvpp/src/vnet/ethernet/arp.c:1514
#13 0x00007f092a020c4f in dispatch_node (vm=0x7f08bc30fd40,
node=0x7f08bcc4dc80, type=VLIB_NODE_TYPE_INTERNAL,
dispatch_state=VLIB_NODE_STATE_POLLING,
frame=0x7f08be1835c0, last_time_stamp=3193892880085078) at
/development/libvpp/src/vlib/main.c:1207
#14 0x00007f092a02140a in dispatch_pending_node (vm=0x7f08bc30fd40,
pending_frame_index=2, last_time_stamp=3193892880085078)
at /development/libvpp/src/vlib/main.c:1375
#15 0x00007f092a02304e in vlib_main_or_worker_loop (vm=0x7f08bc30fd40,
is_main=0) at /development/libvpp/src/vlib/main.c:1826
#16 0x00007f092a0238d1 in vlib_worker_loop (vm=0x7f08bc30fd40) at
/development/libvpp/src/vlib/main.c:1934
#17 0x00007f092a062306 in vlib_worker_thread_fn (arg=0x7f08b9749040) at
/development/libvpp/src/vlib/threads.c:1754
#18 0x00007f0929ce3600 in clib_calljmp () from
/usr/local/lib/libvppinfra.so.1.0.1
#19 0x00007f085fffeec0 in ?? ()
#20 0x00007f092a05cb92 in vlib_worker_thread_bootstrap_fn (arg=0x7f08b9749040)
at /development/libvpp/src/vlib/threads.c:573
---Type <return> to continue, or q <return> to quit---q
Quit
(gdb) thread 10
[Switching to thread 10 (Thread 0x7f085ffff700 (LWP 260))]
#0 0x00007f0935cb86c2 in __GI___waitpid (pid=7605,
stat_loc=stat_loc@entry=0x7f08bc73db18, options=options@entry=0) at
../sysdeps/unix/sysv/linux/waitpid.c:30
30 ../sysdeps/unix/sysv/linux/waitpid.c: No such file or directory.
(gdb) fr 11
#11 0x00007f092aa096bf in arp_mk_reply (vnm=0x7f092b5a05a0 <vnet_main>,
p0=0x1001e349c0, sw_if_index0=8, if_addr0=0x7f08bbe399f4, arp0=0x1001e34b0e,
eth_rx=0x1001e34b00) at /development/libvpp/src/vnet/ethernet/arp.c:1206
1206 /development/libvpp/src/vnet/ethernet/arp.c: No such file or directory.
(gdb) info locals
hw_if0 = 0x7f08bcbd3538
rewrite0 = 0x7f08be193740 ""
rewrite0_len = 14 '\016'
eth_tx = 0x1001e34b00
next0 = 1
__FUNCTION__ = "arp_mk_reply"
(gdb) p *hw_if0
$1 = {name = 0x7f08bd513560 "ifp-0/0/3", flags =
(VNET_HW_INTERFACE_FLAG_LINK_UP | VNET_HW_INTERFACE_FLAG_SUPPORTS_INT_MODE),
link_speed = 0,
hw_address = 0x7f08bbcedc20 "\017", output_node_index = 619, tx_node_index =
618, dev_class_index = 10, dev_instance = 2, hw_class_index = 25, hw_instance =
7,
hw_if_index = 8, sw_if_index = 8, output_node_next_index = 8,
max_rate_bits_per_sec = 0, min_supported_packet_bytes = 64,
max_supported_packet_bytes = 9216,
min_packet_bytes = 64, max_packet_bytes = 9216,
sub_interface_sw_if_index_by_id = 0x0, l2_if_count = 0, bond_info = 0x0,
input_node_index = 218,
input_node_thread_index_by_queue = 0x7f08bbcedd50, rx_mode_by_queue =
0x7f08bbcedb40 "\002", default_rx_mode = VNET_HW_INTERFACE_RX_MODE_POLLING,
dq_runtime_index_by_queue = 0x7f08bd513d60, numa_node = 0 '\000', padding =
"\000\000"}
(gdb) p ((vec_header_t *)$2) -1
$3 = (vec_header_t *) 0x7f08bbcedc18
(gdb) p *$3
$4 = {len = 15, dlmalloc_header_offset = 15, vector_data = 0x7f08bbcedc20
"\017"}
(gdb) x/6b hw_if0->hw_address
0x7f08bbcedc20: 15 0 0 0 15 0
(gdb) p hw_if0->hw_address
$5 = (u8 *) 0x7f08bbcedc20 "\017"
(gdb) p hw_if0->hw_address[0]
$6 = 15 '\017'
(gdb) x/15b hw_if0->hw_address
0x7f08bbcedc20: 15 0 0 0 15 0 0 0
0x7f08bbcedc28: 15 0 0 0 15 0 0
(gdb) p sw_if_index0
$7 = 8
(gdb) p arp0
$8 = (ethernet_arp_header_t *) 0x1001e34b0e
(gdb) p *arp0
$9 = {l2_type = 256, l3_type = 8, n_l2_address_bytes = 6 '\006',
n_l3_address_bytes = 4 '\004', opcode = 512, {ip4_over_ethernet = {{mac = {{
bytes = "\017\000\000\000\017", u = {first_4 = 15, last_2 = 15}}},
ip4 = {data = "\"\000\000\003", data_u32 = 50331682, as_u8 = "\"\000\000\003",
as_u16 = {34, 768}, as_u32 = 50331682}}, {mac = {{bytes =
"\000\022\001\000\000\001", u = {first_4 = 70144, last_2 = 256}}}, ip4 = {data
= "\"\000\000\004",
data_u32 = 67108898, as_u8 = "\"\000\000\004", as_u16 = {34, 1024},
as_u32 = 67108898}}}, data = 0x1001e34b16 "\017"}}
(gdb) p vnm->interface_main.sw_interfaces[8]
$10 = {type = VNET_SW_INTERFACE_TYPE_HARDWARE, flags =
VNET_SW_INTERFACE_FLAG_ADMIN_UP, sw_if_index = 8, sup_sw_if_index = 8,
unnumbered_sw_if_index = 0,
hw_if_index = 8, mtu = {0, 1500, 1500, 1500}, sub = {id = 0, eth =
{outer_vlan_id = 0, inner_vlan_id = 0, {raw_flags = 0, flags = {no_tags = 0,
one_tag = 0,
two_tags = 0, dot1ad = 0, exact_match = 0, default_sub = 0,
outer_vlan_id_any = 0, inner_vlan_id_any = 0}}}}, p2p = {id = 0, pool_index = 0,
client_mac = "\000\000\000\000\000"}, flood_class = VNET_FLOOD_CLASS_NORMAL}
Thanks,
Rajith
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#16840): https://lists.fd.io/g/vpp-dev/message/16840
Mute This Topic: https://lists.fd.io/mt/75174006/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
