I put an assert just to be sure that virtio-input got a frame with the
ETH_INPUT_FRAME_F_SINGLE_SW_IF_IDX
flag.
Attaching to process 324596
[New LWP 324612]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0x00007fc5c21cf1dc in virtio_device_input_inline (vm=0x7fc580e0b740,
node=0x7fc581491f40, frame=0x0, vif=0x7fc58175f900,
qid=0, type=VIRTIO_IF_TYPE_TAP) at
/home/zstas/vpp/src/vnet/devices/virtio/node.c:447
447 /home/zstas/vpp/src/vnet/devices/virtio/node.c: No such file or
directory.
Continuing.
Thread 1 "vpp_main" received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1 0x00007fc5c0f33859 in __GI_abort () at abort.c:79
#2 0x00000000004072f3 in os_panic () at
/home/zstas/vpp/src/vpp/vnet/main.c:413
#3 0x00007fc5c1283bc9 in debugger () at
/home/zstas/vpp/src/vppinfra/error.c:84
#4 0x00007fc5c128392d in _clib_error (how_to_die=2, function_name=0x0,
line_number=0,
fmt=0x7fc5c22051a7 "%s:%d (%s) assertion `%s' fails") at
/home/zstas/vpp/src/vppinfra/error.c:143
#5 0x00007fc5c21cf4ae in virtio_device_input_gso_inline
(vm=0x7fc580e0b740, node=0x7fc581491f40, frame=0x0,
vif=0x7fc58175ef00, vring=0x7fc5866aedc0, type=VIRTIO_IF_TYPE_TAP,
gso_enabled=0, checksum_offload_enabled=0, packed=0)
at /home/zstas/vpp/src/vnet/devices/virtio/node.c:266
#6 0x00007fc5c21cf1e4 in virtio_device_input_inline (vm=0x7fc580e0b740,
node=0x7fc581491f40, frame=0x0, vif=0x7fc58175ef00,
qid=0, type=VIRTIO_IF_TYPE_TAP) at
/home/zstas/vpp/src/vnet/devices/virtio/node.c:447
#7 0x00007fc5c21cedb1 in virtio_input_node_fn_icl (vm=0x7fc580e0b740,
node=0x7fc581491f40, frame=0x0)
at /home/zstas/vpp/src/vnet/devices/virtio/node.c:472
#8 0x00007fc5c13f3f48 in dispatch_node (vm=0x7fc580e0b740,
node=0x7fc581491f40, type=VLIB_NODE_TYPE_INPUT,
dispatch_state=VLIB_NODE_STATE_POLLING, frame=0x0,
last_time_stamp=388854079070548) at /home/zstas/vpp/src/vlib/main.c:961
#9 0x00007fc5c13ef276 in vlib_main_or_worker_loop (vm=0x7fc580e0b740,
is_main=1) at /home/zstas/vpp/src/vlib/main.c:1556
#10 0x00007fc5c13f14ba in vlib_main_loop (vm=0x7fc580e0b740) at
/home/zstas/vpp/src/vlib/main.c:1715
#11 0x00007fc5c13f129e in vlib_main (vm=0x7fc580e0b740,
input=0x7fc57c8b9fa8) at /home/zstas/vpp/src/vlib/main.c:2009
#12 0x00007fc5c145cd5e in thread0 (arg=140486247495488) at
/home/zstas/vpp/src/vlib/unix/main.c:656
#13 0x00007fc5c129cb48 in clib_calljmp () at
/home/zstas/vpp/src/vppinfra/longjmp.S:123
#14 0x00007ffcb8196770 in ?? ()
#15 0x00007fc5c145c94c in vlib_unix_main (argc=82, argv=0x1873450) at
/home/zstas/vpp/src/vlib/unix/main.c:736
#16 0x00000000004066bf in main (argc=82, argv=0x1873450) at
/home/zstas/vpp/src/vpp/vnet/main.c:342
(gdb) select 5
(gdb) info locals
_f = 0x7fc5867ad040
_n = 25
n_left_to_next = 0
next0 = 4
vnm = 0x7fc5c24a06e0 <vnet_main>
thread_index = 0
n_trace = 0
next_index = 4
hdr_sz = 12
to_next = 0x0
n_rx_packets = 0
n_rx_bytes = 0
mask = 255
last = 27
n_left = 2
bt = {{cacheline0 = 0x7fc57c8b96c0 "", current_data = 0, current_length =
0, flags = 0, flow_id = 0, ref_count = 0 '\000',
buffer_pool_index = 0 '\000', error = 0, next_buffer = 0,
{current_config_index = 0, punt_reason = 0}, opaque = {0, 0, 0,
0, 0, 0, 0, 0, 0, 0}, template_end = 0x7fc57c8b9700 "", second_half =
0x7fc57c8b9700 "", trace_handle = 0,
total_length_not_including_first_buffer = 0, opaque2 = {0 <repeats 14
times>}, headroom = 0x7fc57c8b9740 "",
pre_data = '\000' <repeats 127 times>, data = 0x7fc57c8b97c0
"`\230\213|\305\177"}, as_u8x16 = {'\000' <repeats 15 times>,
'\000' <repeats 15 times>, '\000' <repeats 15 times>, '\000' <repeats
15 times>}, as_u8x32 = {'\000' <repeats 31 times>,
'\000' <repeats 31 times>}, as_u8x64 = {'\000' <repeats 63 times>}}
(gdb) p *_f
$1 = {frame_flags = 6, flags = 1, scalar_offset = 16, vector_offset = 32,
aux_offset = 0, n_vectors = 25,
arguments = 0x7fc5867ad04c "\376\376\376\376\003"}
(gdb)
On Wed, 30 Mar 2022 at 19:35, Stanislav Zaikin via lists.fd.io <zstaseg=
gmail....@lists.fd.io> wrote:
> Looks like I found the issue:
>
> it happens when ethernet-node gets a frame with flag
> ETH_INPUT_FRAME_F_SINGLE_SW_IF_IDX but they're actually packets from
> different interfaces (from virtio-node but some of them are l2 and some of
> them are l3). virtio-input doesn't set this flag, but I thought maybe it
> just wasn't cleared.
>
> After I've added a piece of code which clears the flag in frames - the
> issue was gone:
> diff --git a/src/vlib/main.c b/src/vlib/main.c
> index fd0c1fc5f..e88a0724a 100644
> --- a/src/vlib/main.c
> +++ b/src/vlib/main.c
> @@ -208,6 +208,7 @@ vlib_frame_free (vlib_main_t * vm, vlib_node_runtime_t
> * r, vlib_frame_t * f)
> }
>
> f->frame_flags &= ~(VLIB_FRAME_IS_ALLOCATED | VLIB_FRAME_NO_APPEND);
> + f->flags = 0;
>
> vec_add1 (fs->free_frames, f);
> ASSERT (fs->n_alloc_frames > 0);
> @@ -1163,6 +1164,7 @@ dispatch_pending_node (vlib_main_t * vm, uword
> pending_frame_index,
> /* no new frame has been assigned to this node, use the saved
> one */
> nf->frame = restore_frame;
> f->n_vectors = 0;
> + f->flags = 0;
> }
> else
> {
>
> If I'm correct, I'd like to send 2 patches, this one and another which
> allocates a new frame per interface with the
> ETH_INPUT_FRAME_F_SINGLE_SW_IF_IDX flag in virtio-input node.
> Any feedback is appreciated.
>
> On Tue, 29 Mar 2022 at 22:09, Stanislav Zaikin <zsta...@gmail.com> wrote:
>
>> Hello folks,
>>
>> I'm experiencing a weird issue. I'm running a current master with
>> linux-cp plugin enabled.
>> I create a loopback interface, set it as BVI for a bridge, and create a
>> lcp for the loop. Almost everywhere it's working just fine. Usually, the
>> trace looks like this:
>>
>> virtio-input -> ethernet-input -> ip4-input -> linux-cp-xc-ip4 ->
>> loop20-output -> l2-input -> etc
>>
>> But on 2 machines I observe a crash almost immediately as some traffic
>> comes from the tap interface (which is lcp interface for the loop). It
>> happens in the "l2-input" node. But the tap interface isn't a part of any
>> bridge (and obviously it doesn't have any l2 config, and that's why vpp
>> crashes).
>>
>> I even enabled VLIB_BUFFER_TRACE_TRAJECTORY to check how the trace looks
>> like. I can see that the packet goes directly from "ethernet-input" to
>> "l2-input" (which obviously shouldn't happen).
>>
>> I double-checked the sources of ethernet-input and the only reason could
>> be if in ethernet_main.main_intfs[sw_if_index] there's a flag
>> SUBINT_CONFIG_L2 for an untagged subinterface. I checked that in gdb and I
>> don't have such a flag for my tap interface.
>>
>> Since I run out of ideas on what to check I'm asking for the help of the
>> community. So, any ideas/inputs are much appreciated.
>>
>> --
>> Best regards
>> Stanislav Zaikin
>>
>
>
> --
> Best regards
> Stanislav Zaikin
>
>
>
>
--
Best regards
Stanislav Zaikin
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21155): https://lists.fd.io/g/vpp-dev/message/21155
Mute This Topic: https://lists.fd.io/mt/90117260/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
