Hi Neale,
Probably I didn’t make it clear enough, sorry for that. I just re-produced the
case and captured the back traces for your reference.
There are actually two cases:
First, creating many tunnels with different tunnel endpoints. In this case,
fib_table_entry_special_add would create many fib entries, but all entries have
to be added to a linked list for cover route check then the list will be walked
through by fib_entry_cover_walk.
See back trace for this case:
-------------------- back trace of linked list formation
#0 fib_node_list_push_front (list=13, owner_id=0, type=FIB_NODE_TYPE_ENTRY,
index=11)
at /home/vppshare/kingwel/vpp/build-data/../src/vnet/fib/fib_node_list.c:160
#1 0x00007ffff7390c80 in fib_entry_cover_track (cover=0x7fffb6fea840,
covered=11)
at
/home/vppshare/kingwel/vpp/build-data/../src/vnet/fib/fib_entry_cover.c:38
#2 0x00007ffff738d367 in fib_entry_src_rr_activate (src=0x7fffb5aab734,
fib_entry=0x7fffb6feab58)
at
/home/vppshare/kingwel/vpp/build-data/../src/vnet/fib/fib_entry_src_rr.c:143
#3 0x00007ffff738ac7a in fib_entry_src_action_activate
(fib_entry=0x7fffb6feab58, source=FIB_SOURCE_RR)
at /home/vppshare/kingwel/vpp/build-data/../src/vnet/fib/fib_entry_src.c:916
#4 0x00007ffff7386a43 in fib_entry_create_special (fib_index=0,
prefix=0x7fffb6e888f0, source=FIB_SOURCE_RR,
flags=FIB_ENTRY_FLAG_NONE, dpo=0x7fffb6e88760) at
/home/vppshare/kingwel/vpp/build-data/../src/vnet/fib/fib_entry.c:730
#5 0x00007ffff7371b74 in fib_table_entry_special_dpo_add (fib_index=0,
prefix=0x7fffb6e888f0, source=FIB_SOURCE_RR,
flags=FIB_ENTRY_FLAG_NONE, dpo=0x7fffb6e88760) at
/home/vppshare/kingwel/vpp/build-data/../src/vnet/fib/fib_table.c:313
#6 0x00007ffff7371e0f in fib_table_entry_special_add (fib_index=0,
prefix=0x7fffb6e888f0, source=FIB_SOURCE_RR,
flags=FIB_ENTRY_FLAG_NONE) at
/home/vppshare/kingwel/vpp/build-data/../src/vnet/fib/fib_table.c:390
#7 0x00007fffb359afb2 in vnet_gtpu_add_del_tunnel (a=0x7fffb6e88a50,
sw_if_indexp=0x7fffb6e889dc)
at /home/vppshare/kingwel/vpp/build-data/../src/plugins/gtpu/gtpu.c:490
---------------- back trace of cover walk the long list
#0 fib_node_list_walk (list=13, fn=0x7ffff7390d2f
<fib_entry_cover_walk_node_ptr>, args=0x7fffb6e885f0)
at /home/vppshare/kingwel/vpp/build-data/../src/vnet/fib/fib_node_list.c:357
#1 0x00007ffff7390e04 in fib_entry_cover_walk (cover=0x7fffb550c6cc,
walk=0x7ffff7390e1d <fib_entry_cover_change_one>, args=0x7)
at
/home/vppshare/kingwel/vpp/build-data/../src/vnet/fib/fib_entry_cover.c:104
#2 0x00007ffff7390f0a in fib_entry_cover_change_notify (cover_index=0,
covered=7)
at
/home/vppshare/kingwel/vpp/build-data/../src/vnet/fib/fib_entry_cover.c:158
#3 0x00007ffff73717d7 in fib_table_post_insert_actions
(fib_table=0x7fffb5707cc0, prefix=0x7fffb6e888f0, fib_entry_index=7)
at /home/vppshare/kingwel/vpp/build-data/../src/vnet/fib/fib_table.c:202
#4 0x00007ffff73718f7 in fib_table_entry_insert (fib_table=0x7fffb5707cc0,
prefix=0x7fffb6e888f0, fib_entry_index=7)
at /home/vppshare/kingwel/vpp/build-data/../src/vnet/fib/fib_table.c:239
#5 0x00007ffff7371b8d in fib_table_entry_special_dpo_add (fib_index=0,
prefix=0x7fffb6e888f0, source=FIB_SOURCE_RR,
flags=FIB_ENTRY_FLAG_NONE, dpo=0x7fffb6e88760) at
/home/vppshare/kingwel/vpp/build-data/../src/vnet/fib/fib_table.c:317
#6 0x00007ffff7371e0f in fib_table_entry_special_add (fib_index=0,
prefix=0x7fffb6e888f0, source=FIB_SOURCE_RR,
flags=FIB_ENTRY_FLAG_NONE) at
/home/vppshare/kingwel/vpp/build-data/../src/vnet/fib/fib_table.c:390
#7 0x00007fffb359afb2 in vnet_gtpu_add_del_tunnel (a=0x7fffb6e88a50,
sw_if_indexp=0x7fffb6e889dc)
at /home/vppshare/kingwel/vpp/build-data/../src/plugins/gtpu/gtpu.c:490
Then the second case is, creating many tunnels but with one (or very few, which
I think is the classic GTP case) tunnel endpoint. Thus, only one fib entry will
created, but fib_entry_child_add will create a list for this fib entry to
accommodate all fib node children which is the GTP fib node. This linked list
will be walked through by next fib_entry_special_add for restacking the dpo.
See back trace:
#0 gtpu_tunnel_restack_dpo (t=0x7fffb6c45d00) at
/home/vppshare/kingwel/vpp/build-data/../src/plugins/gtpu/gtpu.c:139
#1 0x00007fffb3599b15 in gtpu_tunnel_back_walk (node=0x7fffb6c45d48,
ctx=0x7fffb6dc32b8)
at /home/vppshare/kingwel/vpp/build-data/../src/plugins/gtpu/gtpu.c:165
#2 0x00007ffff73811ed in fib_node_back_walk_one (ptr=0x7fffb6e86590,
ctx=0x7fffb6dc32b8)
at /home/vppshare/kingwel/vpp/build-data/../src/vnet/fib/fib_node.c:161
#3 0x00007ffff737c73f in fib_walk_advance (fwi=0) at
/home/vppshare/kingwel/vpp/build-data/../src/vnet/fib/fib_walk.c:353
#4 0x00007ffff737d18c in fib_walk_sync (parent_type=FIB_NODE_TYPE_ENTRY,
parent_index=7, ctx=0x7fffb6e86620)
at /home/vppshare/kingwel/vpp/build-data/../src/vnet/fib/fib_walk.c:772
#5 0x00007ffff7386acd in fib_entry_post_update_actions
(fib_entry=0x7fffb54f88c4, source=FIB_SOURCE_RR,
old_flags=FIB_ENTRY_FLAG_NONE) at
/home/vppshare/kingwel/vpp/build-data/../src/vnet/fib/fib_entry.c:749
#6 0x00007ffff7386ba9 in fib_entry_source_change (fib_entry=0x7fffb54f88c4,
old_source=FIB_SOURCE_RR, new_source=FIB_SOURCE_RR)
at /home/vppshare/kingwel/vpp/build-data/../src/vnet/fib/fib_entry.c:795
#7 0x00007ffff7386c3c in fib_entry_special_add (fib_entry_index=7,
source=FIB_SOURCE_RR, flags=FIB_ENTRY_FLAG_NONE,
dpo=0x7fffb6e86760) at
/home/vppshare/kingwel/vpp/build-data/../src/vnet/fib/fib_entry.c:811
#8 0x00007ffff7371bdc in fib_table_entry_special_dpo_add (fib_index=0,
prefix=0x7fffb6e868f0, source=FIB_SOURCE_RR,
flags=FIB_ENTRY_FLAG_NONE, dpo=0x7fffb6e86760) at
/home/vppshare/kingwel/vpp/build-data/../src/vnet/fib/fib_table.c:325
#9 0x00007ffff7371e0f in fib_table_entry_special_add (fib_index=0,
prefix=0x7fffb6e868f0, source=FIB_SOURCE_RR,
flags=FIB_ENTRY_FLAG_NONE) at
/home/vppshare/kingwel/vpp/build-data/../src/vnet/fib/fib_table.c:390
#10 0x00007fffb359b006 in vnet_gtpu_add_del_tunnel (a=0x7fffb6e86a50,
sw_if_indexp=0x7fffb6e869dc)
at /home/vppshare/kingwel/vpp/build-data/../src/plugins/gtpu/gtpu.c:490
--------------------
The tentative solution is simple: comment fib_table_entry_special_add,
fib_entry_child_add, gtpu_tunnel_restack_dpo. It is ok for a gtp tunnel, which
might not be so critical for missing route change notification. Not so good at
fib part of vPP code, but we really appreciate if someone can point out a
better solution.
Regards,
Kingwel
-----Original Message-----
From: Neale Ranns (nranns) [mailto:[email protected]]
Sent: Thursday, April 19, 2018 9:06 PM
To: Kingwel Xie <[email protected]>; xyxue <[email protected]>
Cc: [email protected]
Subject: Re: [vpp-dev] questions in configuring tunnel
Hi Kingwei,
[nr] if you skip this then the tunnels are not part of the FIB graph and hence
any updates in the forwarding to the tunnel’s destination will go unnoticed and
hence you potentially black hole the tunnel traffic indefinitely (since the
tunnel is not re-stacked). It is a linked list, but apart from the pool
allocation of the list element, the list element insertion is O(1), no?
[kingwel] You are right that the update will not be noticed, but we think it is
acceptable for a p2p tunnel interface. The list element itself is ok when being
inserted, but the following restack operation will walk through all inserted
elements. This is the point I’m talking about.
restacking will indeed walk all of the child objects of a parent, but this is
an operation that occurs only when the forwarding of that parent changes (which
in this case is the route to the tunnel’s destination) and this walk is done
asynchronously, since the child uses a recursive path. You should not see these
walks occurring whilst you are creating tunnels (please confirm) so they should
not affect the tunnel setup time.
/neale
-=-=-=-=-=-=-=-=-=-=-=-
Links:
You receive all messages sent to this group.
View/Reply Online (#9004): https://lists.fd.io/g/vpp-dev/message/9004
View All Messages In Topic (11): https://lists.fd.io/g/vpp-dev/topic/17543966
Mute This Topic: https://lists.fd.io/mt/17543966/21656
New Topic: https://lists.fd.io/g/vpp-dev/post
Change Your Subscription: https://lists.fd.io/g/vpp-dev/editsub/21656
Group Home: https://lists.fd.io/g/vpp-dev
Contact Group Owner: [email protected]
Terms of Service: https://lists.fd.io/static/tos
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub
-=-=-=-=-=-=-=-=-=-=-=-
