Re: [vpp-dev] vpp crashes on deleting route 0.0.0.0/0 via interface #vpp

Wed, 15 Jan 2020 07:16:43 -0800

Hi!
 
2Neale: my scenario is taking MPLS traffic, routing it to a specific ip table based on MPLS label. Then IP packets should be routed within that specific table with some routes pointing to virtual devices, including a default. The backtrace and reason for firing assert is the same as in this gre scenario. And it works for table 0, but I wonder why it doesn't work for a non-zero ip table.
 
Unfotunately, Aleksander's patch did not help:
```
DBGvpp# ip table add 10
DBGvpp# create gre tunnel src 1.1.1.1 dst 2.2.2.2
gre0
DBGvpp# ip route add 0.0.0.0/0 table 10 via gre0
DBGvpp# sh ip fib table 10 0.0.0.0/0
ipv4-VRF:10, fib_index:1, flow hash:[src dst sport dport proto ] epoch:0 flags:none locks:[CLI:1, ]
0.0.0.0/0 fib:1 index:7 locks:3
CLI refs:1 entry-flags:attached,import, src-flags:added,contributing,active,
path-list:[16] locks:2 flags:shared, uPRF-list:12 len:1 itfs:[1, ]
path:[16] pl-index:16 ip4 weight=1 pref=0 attached:
gre0
 
default-route refs:1 entry-flags:drop, src-flags:added,
path-list:[11] locks:1 flags:drop, uPRF-list:7 len:0 itfs:[]
path:[11] pl-index:11 ip4 weight=1 pref=0 special: cfg-flags:drop,
[@0]: dpo-drop ip4
 
forwarding: unicast-ip4-chain
[@0]: dpo-load-balance: [proto:ip4 index:9 buckets:1 uRPF:12 to:[0:0]]
[0] [@0]: dpo-drop ip4
DBGvpp# ip route del 0.0.0.0/0 table 10 via gre0
/home/elantsev/vpp/src/vnet/fib/fib_attached_export.c:367 (fib_attached_export_purge) assertion `NULL != fed' fails
 
Thread 1 "vpp_main" received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
51 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1 0x00007ffff5ac9801 in __GI_abort () at abort.c:79
#2 0x000055555555be0b in os_panic () at /home/elantsev/vpp/src/vpp/vnet/main.c:355
#3 0x00007ffff5eacde9 in debugger () at /home/elantsev/vpp/src/vppinfra/error.c:84
#4 0x00007ffff5ead1b8 in _clib_error (how_to_die=2, function_name=0x0, line_number=0, fmt=0x7ffff7743bb8 "%s:%d (%s) assertion `%s' fails") at /home/elantsev/vpp/src/vppinfra/error.c:143
#5 0x00007ffff74cbbe9 in fib_attached_export_purge (fib_entry=0x7fffb83886b0) at /home/elantsev/vpp/src/vnet/fib/fib_attached_export.c:367
#6 0x00007ffff74919de in fib_entry_post_flag_update_actions (fib_entry=0x7fffb83886b0, old_flags=(FIB_ENTRY_FLAG_ATTACHED | FIB_ENTRY_FLAG_IMPORT)) at /home/elantsev/vpp/src/vnet/fib/fib_entry.c:674
#7 0x00007ffff7491a3c in fib_entry_post_install_actions (fib_entry=0x7fffb83886b0, source=FIB_SOURCE_DEFAULT_ROUTE, old_flags=(FIB_ENTRY_FLAG_ATTACHED | FIB_ENTRY_FLAG_IMPORT))
at /home/elantsev/vpp/src/vnet/fib/fib_entry.c:709
#8 0x00007ffff7491d78 in fib_entry_post_update_actions (fib_entry=0x7fffb83886b0, source=FIB_SOURCE_DEFAULT_ROUTE, old_flags=(FIB_ENTRY_FLAG_ATTACHED | FIB_ENTRY_FLAG_IMPORT))
at /home/elantsev/vpp/src/vnet/fib/fib_entry.c:804
#9 0x00007ffff74923f9 in fib_entry_source_removed (fib_entry=0x7fffb83886b0, old_flags=(FIB_ENTRY_FLAG_ATTACHED | FIB_ENTRY_FLAG_IMPORT)) at /home/elantsev/vpp/src/vnet/fib/fib_entry.c:992
#10 0x00007ffff74925e7 in fib_entry_path_remove (fib_entry_index=7, source=FIB_SOURCE_CLI, rpaths=0x7fffb83a34b0) at /home/elantsev/vpp/src/vnet/fib/fib_entry.c:1072
#11 0x00007ffff747980b in fib_table_entry_path_remove2 (fib_index=1, prefix=0x7fffb8382b00, source=FIB_SOURCE_CLI, rpaths=0x7fffb83a34b0) at /home/elantsev/vpp/src/vnet/fib/fib_table.c:680
#12 0x00007ffff6fb870c in vnet_ip_route_cmd (vm=0x7ffff66b6680 <vlib_global_main>, main_input=0x7fffb8382f00, cmd=0x7fffb50373b8) at /home/elantsev/vpp/src/vnet/ip/lookup.c:449
#13 0x00007ffff63d402f in vlib_cli_dispatch_sub_commands (vm=0x7ffff66b6680 <vlib_global_main>, cm=0x7ffff66b68b0 <vlib_global_main+560>, input=0x7fffb8382f00, parent_command_index=431)
at /home/elantsev/vpp/src/vlib/cli.c:568
#14 0x00007ffff63d3ead in vlib_cli_dispatch_sub_commands (vm=0x7ffff66b6680 <vlib_global_main>, cm=0x7ffff66b68b0 <vlib_global_main+560>, input=0x7fffb8382f00, parent_command_index=0)
at /home/elantsev/vpp/src/vlib/cli.c:528
#15 0x00007ffff63d4434 in vlib_cli_input (vm=0x7ffff66b6680 <vlib_global_main>, input=0x7fffb8382f00, function=0x7ffff646dc89 <unix_vlib_cli_output>, function_arg=0) at /home/elantsev/vpp/src/vlib/cli.c:667
#16 0x00007ffff647476d in unix_cli_process_input (cm=0x7ffff66b7020 <unix_cli_main>, cli_file_index=0) at /home/elantsev/vpp/src/vlib/unix/cli.c:2572
#17 0x00007ffff647540e in unix_cli_process (vm=0x7ffff66b6680 <vlib_global_main>, rt=0x7fffb8342000, f=0x0) at /home/elantsev/vpp/src/vlib/unix/cli.c:2688
#18 0x00007ffff64161d4 in vlib_process_bootstrap (_a=140736272894320) at /home/elantsev/vpp/src/vlib/main.c:1475
#19 0x00007ffff5eccfbc in clib_calljmp () at /home/elantsev/vpp/src/vppinfra/longjmp.S:123
#20 0x00007fffb78d8940 in ?? ()
#21 0x00007ffff64162dc in vlib_process_startup (vm=0x0, p=0x8, f=0x7ffff66b6680 <vlib_global_main>) at /home/elantsev/vpp/src/vlib/main.c:1497
Backtrace stopped: previous frame inner to this frame (corrupt stack?)
(gdb)
```
 
15.01.2020, 01:54, "Neale Ranns via Lists.Fd.Io" <[email protected]>:

Hi,

 

Thanks for the bug report, I’ll fix the crash.

 

A question for you.

DBGvpp# ip route add 0.0.0.0/0 table 10 via gre0
 

Says “all destinations in table 10 are reachable via an interface in table 0”. It implies therefore that all addresses in table 10 refer to the same device in table 0, i.e. there is no difference, or separation, between table 10 and 0. In other words it’s not something one normally does 😊

What’s your use case for this config?

 

Thanks,

Neale

 

 

From: <[email protected]> on behalf of "[email protected]" <[email protected]>
Date: Tuesday 14 January 2020 at 17:35
To: "[email protected]" <[email protected]>
Subject: [vpp-dev] vpp crashes on deleting route 0.0.0.0/0 via interface #vpp

 

Hello Everyone!

I've encountered an issue with deleting route to 0.0.0.0/0 via some virtual interface: vpp crashed with a SIGABRT. This issue can be reproduced with gre interface on the current master 1c6486f7b8a00a1358d5c8f4ea1d874073bbcd6c:

DBGvpp# ip table add 10
DBGvpp# create gre tunnel src 1.1.1.1 dst 2.2.2.2
gre0
 
DBGvpp# ip route add 0.0.0.0/0 table 10 via gre0
DBGvpp# sh ip fib table 10
ipv4-VRF:10, fib_index:1, flow hash:[src dst sport dport proto ] epoch:0 flags:none locks:[CLI:1, ]
0.0.0.0/0
  unicast-ip4-chain
  [@0]: dpo-load-balance: [proto:ip4 index:9 buckets:1 uRPF:12 to:[0:0]]
    [0] [@0]: dpo-drop ip4
0.0.0.0/32
  unicast-ip4-chain
  [@0]: dpo-load-balance: [proto:ip4 index:10 buckets:1 uRPF:8 to:[0:0]]
    [0] [@0]: dpo-drop ip4
224.0.0.0/4
  unicast-ip4-chain
  [@0]: dpo-load-balance: [proto:ip4 index:12 buckets:1 uRPF:10 to:[0:0]]
    [0] [@0]: dpo-drop ip4
240.0.0.0/4
  unicast-ip4-chain
  [@0]: dpo-load-balance: [proto:ip4 index:11 buckets:1 uRPF:9 to:[0:0]]
    [0] [@0]: dpo-drop ip4
255.255.255.255/32
  unicast-ip4-chain
  [@0]: dpo-load-balance: [proto:ip4 index:13 buckets:1 uRPF:11 to:[0:0]]
    [0] [@0]: dpo-drop ip4
 
/home/elantsev/vpp/src/vnet/fib/fib_attached_export.c:367 (fib_attached_export_purge) assertion `NULL != fed' fails
 
Thread 1 "vpp_main" received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
51       ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
 
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1  0x00007ffff5ac9801 in __GI_abort () at abort.c:79
#2  0x000055555555be0b in os_panic () at /home/elantsev/vpp/src/vpp/vnet/main.c:355
#3  0x00007ffff5eacde9 in debugger () at /home/elantsev/vpp/src/vppinfra/error.c:84
#4  0x00007ffff5ead1b8 in _clib_error (how_to_die=2, function_name=0x0, line_number=0, fmt=0x7ffff7743bb8 "%s:%d (%s) assertion `%s' fails") at /home/elantsev/vpp/src/vppinfra/error.c:143
#5  0x00007ffff74cbbe0 in fib_attached_export_purge (fib_entry=0x7fffb4bd2dd0) at /home/elantsev/vpp/src/vnet/fib/fib_attached_export.c:367
#6  0x00007ffff74919de in fib_entry_post_flag_update_actions (fib_entry=0x7fffb4bd2dd0, old_flags=(FIB_ENTRY_FLAG_ATTACHED | FIB_ENTRY_FLAG_IMPORT)) at /home/elantsev/vpp/src/vnet/fib/fib_entry.c:674
#7  0x00007ffff7491a3c in fib_entry_post_install_actions (fib_entry=0x7fffb4bd2dd0, source=FIB_SOURCE_DEFAULT_ROUTE, old_flags=(FIB_ENTRY_FLAG_ATTACHED | FIB_ENTRY_FLAG_IMPORT))
    at /home/elantsev/vpp/src/vnet/fib/fib_entry.c:709
#8  0x00007ffff7491d78 in fib_entry_post_update_actions (fib_entry=0x7fffb4bd2dd0, source=FIB_SOURCE_DEFAULT_ROUTE, old_flags=(FIB_ENTRY_FLAG_ATTACHED | FIB_ENTRY_FLAG_IMPORT))
    at /home/elantsev/vpp/src/vnet/fib/fib_entry.c:804
#9  0x00007ffff74923f9 in fib_entry_source_removed (fib_entry=0x7fffb4bd2dd0, old_flags=(FIB_ENTRY_FLAG_ATTACHED | FIB_ENTRY_FLAG_IMPORT)) at /home/elantsev/vpp/src/vnet/fib/fib_entry.c:992
#10 0x00007ffff74925e7 in fib_entry_path_remove (fib_entry_index=7, source=FIB_SOURCE_CLI, rpaths=0x7fffb83a3520) at /home/elantsev/vpp/src/vnet/fib/fib_entry.c:1072
#11 0x00007ffff747980b in fib_table_entry_path_remove2 (fib_index=1, prefix=0x7fffb8382b00, source=FIB_SOURCE_CLI, rpaths=0x7fffb83a3520) at /home/elantsev/vpp/src/vnet/fib/fib_table.c:680
#12 0x00007ffff6fb870c in vnet_ip_route_cmd (vm=0x7ffff66b6680 <vlib_global_main>, main_input=0x7fffb8382f00, cmd=0x7fffb50373b8) at /home/elantsev/vpp/src/vnet/ip/lookup.c:449
#13 0x00007ffff63d402f in vlib_cli_dispatch_sub_commands (vm=0x7ffff66b6680 <vlib_global_main>, cm=0x7ffff66b68b0 <vlib_global_main+560>, input=0x7fffb8382f00, parent_command_index=431)
    at /home/elantsev/vpp/src/vlib/cli.c:568
#14 0x00007ffff63d3ead in vlib_cli_dispatch_sub_commands (vm=0x7ffff66b6680 <vlib_global_main>, cm=0x7ffff66b68b0 <vlib_global_main+560>, input=0x7fffb8382f00, parent_command_index=0)
    at /home/elantsev/vpp/src/vlib/cli.c:528
#15 0x00007ffff63d4434 in vlib_cli_input (vm=0x7ffff66b6680 <vlib_global_main>, input=0x7fffb8382f00, function=0x7ffff646dc89 <unix_vlib_cli_output>, function_arg=0) at /home/elantsev/vpp/src/vlib/cli.c:667
#16 0x00007ffff647476d in unix_cli_process_input (cm=0x7ffff66b7020 <unix_cli_main>, cli_file_index=0) at /home/elantsev/vpp/src/vlib/unix/cli.c:2572
#17 0x00007ffff647540e in unix_cli_process (vm=0x7ffff66b6680 <vlib_global_main>, rt=0x7fffb8342000, f=0x0) at /home/elantsev/vpp/src/vlib/unix/cli.c:2688
#18 0x00007ffff64161d4 in vlib_process_bootstrap (_a=140736272894320) at /home/elantsev/vpp/src/vlib/main.c:1475
#19 0x00007ffff5eccfbc in clib_calljmp () at /home/elantsev/vpp/src/vppinfra/longjmp.S:123
#20 0x00007fffb78d8940 in ?? ()
#21 0x00007ffff64162dc in vlib_process_startup (vm=0x0, p=0x8, f=0x7ffff66b6680 <vlib_global_main>) at /home/elantsev/vpp/src/vlib/main.c:1497
Backtrace stopped: previous frame inner to this frame (corrupt stack?)
 
 
#4  0x00007ffff5ead1b8 in _clib_error (how_to_die=2, function_name=0x0, line_number=0, fmt=0x7ffff7743bb8 "%s:%d (%s) assertion `%s' fails") at /home/elantsev/vpp/src/vppinfra/error.c:143
        msg = 0x0
        va = {{gp_offset = 48, fp_offset = 48, overflow_arg_area = 0x7fffb8382710, reg_save_area = 0x7fffb8382630}}
#5  0x00007ffff74cbbe0 in fib_attached_export_purge (fib_entry=0x7fffb4bd2dd0) at /home/elantsev/vpp/src/vnet/fib/fib_attached_export.c:367
        fed = 0x0
        index = 0
        import_index = 0x0
        export_entry = 0x7fffb4bd2bd8
        import = 0x7fffb818d9b8
        export = 0x7fffb83a3620
        fed = 0x7fffb83a3620
        __FUNCTION__ = "fib_attached_export_purge"
#6  0x00007ffff74919de in fib_entry_post_flag_update_actions (fib_entry=0x7fffb4bd2dd0, old_flags=(FIB_ENTRY_FLAG_ATTACHED | FIB_ENTRY_FLAG_IMPORT)) at /home/elantsev/vpp/src/vnet/fib/fib_entry.c:674
        fei = 7
        is_import = 0
        was_import = 16
        is_attached = -146163533
        was_attached = 32767
#7  0x00007ffff7491a3c in fib_entry_post_install_actions (fib_entry=0x7fffb4bd2dd0, source=FIB_SOURCE_DEFAULT_ROUTE, old_flags=(FIB_ENTRY_FLAG_ATTACHED | FIB_ENTRY_FLAG_IMPORT))
    at /home/elantsev/vpp/src/vnet/fib/fib_entry.c:709
No locals.
#8  0x00007ffff7491d78 in fib_entry_post_update_actions (fib_entry=0x7fffb4bd2dd0, source=FIB_SOURCE_DEFAULT_ROUTE, old_flags=(FIB_ENTRY_FLAG_ATTACHED | FIB_ENTRY_FLAG_IMPORT))
    at /home/elantsev/vpp/src/vnet/fib/fib_entry.c:804
        bw_ctx = {fnbw_reason = FIB_NODE_BW_REASON_FLAG_EVALUATE, fnbw_flags = FIB_NODE_BW_FLAG_NONE, fnbw_depth = 1}
#9  0x00007ffff74923f9 in fib_entry_source_removed (fib_entry=0x7fffb4bd2dd0, old_flags=(FIB_ENTRY_FLAG_ATTACHED | FIB_ENTRY_FLAG_IMPORT)) at /home/elantsev/vpp/src/vnet/fib/fib_entry.c:992
        bsrc = 0x7fffb83a2df0
        best_source = FIB_SOURCE_DEFAULT_ROUTE
#10 0x00007ffff74925e7 in fib_entry_path_remove (fib_entry_index=7, source=FIB_SOURCE_CLI, rpaths=0x7fffb83a3520) at /home/elantsev/vpp/src/vnet/fib/fib_entry.c:1072
        sflag = FIB_ENTRY_SRC_FLAG_NONE
        best_source = FIB_SOURCE_CLI
        bflags = (FIB_ENTRY_FLAG_ATTACHED | FIB_ENTRY_FLAG_IMPORT)
        fib_entry = 0x7fffb4bd2dd0
        bsrc = 0x7fffb83a2df0
        __FUNCTION__ = "fib_entry_path_remove"

Can you please provide some hints/fixes?

,

 
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15183): https://lists.fd.io/g/vpp-dev/message/15183
Mute This Topic: https://lists.fd.io/mt/69689561/21656
Mute #vpp: https://lists.fd.io/mk?hashtag=vpp&subid=1480452
Group Owner: [email protected]
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

Reply via email to