*** This bug is a duplicate of bug 1958643 ***
https://bugs.launchpad.net/bugs/1958643
Thanks @Stanislav for the confirmation, will close it as Duplicate.
** This bug has been marked a duplicate of bug 1958643
Unicast RA messages for a VM are filtered out by ovs rules
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2046196
Title:
Unicast ICMPv6 Router Advertisement packets to VM's dropped by OVS
firewall driver
Status in neutron:
Incomplete
Bug description:
When use Open vSwitch Firewall Driver we noticed that ICMPv6 RA
unicast packets are not reaching the VM.
=========================================Troubleshooting
flow:===========================================
1) Catch ICMPv6 RA package on physical hypervisor bonding interface:
#tcpdump -XXvpnei bond0 -Q in -c 1 "icmp6[0] = 134 and ether host
fa:16:3e:68:e8:19"
dropped privs to tcpdump
tcpdump: listening on bond0, link-type EN10MB (Ethernet), capture size 262144
bytes
21:56:43.669691 26:28:b0:96:c0:c7 > fa:16:3e:68:e8:19, ethertype 802.1Q
(0x8100), length 162: vlan 3254, p 0, ethertype IPv6, (flowlabel 0x7cc6b, hlim
255, next-header ICMPv6 (58) payload length: 104) fe80::2428:b0ff:fe96:c0c7 >
fe80::f816:3eff:fe68:e819: [icmp6 sum ok] ICMP6, router advertisement, length
104
hop limit 64, Flags [managed], pref medium, router lifetime 6s,
reachable time 0ms, retrans timer 0ms
prefix info option (3), length 32 (4): 2a05:fc1:200::/64, Flags
[onlink, auto], valid time 2592000s, pref. time 14400s
rdnss option (25), length 40 (5): lifetime 2s, addr: 2a05:fc1::2
addr: 2a05:fc1::3
source link-address option (1), length 8 (1): 26:28:b0:96:c0:c7
advertisement interval option (7), length 8 (1): 2000ms
0x0000: fa16 3e68 e819 2628 b096 c0c7 8100 0cb6 ..>h..&(........
0x0010: 86dd 6007 cc6b 0068 3aff fe80 0000 0000 ..`..k.h:.......
0x0020: 0000 2428 b0ff fe96 c0c7 fe80 0000 0000 ..$(............
0x0030: 0000 f816 3eff fe68 e819 8600 10d2 4080 ....>..h......@.
0x0040: 0006 0000 0000 0000 0000 0304 40c0 0027 ............@..'
0x0050: 8d00 0000 3840 0000 0000 2a05 0fc1 0200 ....8@....*.....
0x0060: 0000 0000 0000 0000 0000 1905 0000 0000 ................
0x0070: 0002 2a05 0fc1 0000 0000 0000 0000 0000 ..*.............
0x0080: 0002 2a05 0fc1 0000 0000 0000 0000 0000 ..*.............
0x0090: 0003 0101 2628 b096 c0c7 0701 0000 0000 ....&(..........
0x00a0: 07d0 ..
2) Trace the package using "ofproto/trace":
#ovs-appctl ofproto/trace br-int in_port=1
fa163e68e8192628b096c0c781000cb686dd6007cc6b00683afffe800000000000002428b0fffe96c0c7fe80000000000000f8163efffe68e819860010d2408000060000000000000000030440c000278d0000003840000000002a050fc102000000000000000000000019050000000000022a050fc10000000000000000000000022a050fc100000000000000000000000301012628b096c0c707010000000007d0
Flow:
icmp6,in_port=1,dl_vlan=3254,dl_vlan_pcp=0,vlan_tci1=0x0000,dl_src=26:28:b0:96:c0:c7,dl_dst=fa:16:3e:68:e8:19,ipv6_src=fe80::2428:b0ff:fe96:c0c7,ipv6_dst=fe80::f816:3eff:fe68:e819,ipv6_label=0x7cc6b,nw_tos=0,nw_ecn=0,nw_ttl=255,icmp_type=134,icmp_code=0
bridge("br-int")
----------------
0. in_port=1,dl_vlan=3254, priority 3, cookie 0x4ef408376e507615
set_field:4097->vlan_vid
goto_table:60
60. dl_vlan=1,dl_dst=fa:16:3e:68:e8:19, priority 90, cookie 0x4ef408376e507615
set_field:0x9e->reg5
set_field:0x1->reg6
pop_vlan
resubmit(,81)
81. ct_state=-trk,ipv6,reg5=0x9e, priority 90, cookie 0x4ef408376e507615
ct(table=82,zone=NXM_NX_REG6[0..15])
drop
-> A clone of the packet is forked to recirculate. The forked pipeline
will be resumed at table 82.
-> Sets the packet to an untracked state, and clears all the conntrack
fields.
Final flow:
icmp6,reg5=0x9e,reg6=0x1,in_port=1,vlan_tci=0x0000,dl_src=26:28:b0:96:c0:c7,dl_dst=fa:16:3e:68:e8:19,ipv6_src=fe80::2428:b0ff:fe96:c0c7,ipv6_dst=fe80::f816:3eff:fe68:e819,ipv6_label=0x7cc6b,nw_tos=0,nw_ecn=0,nw_ttl=255,icmp_type=134,icmp_code=0
Megaflow:
recirc_id=0,ct_state=-trk,eth,icmp6,in_port=1,dl_vlan=3254,dl_vlan_pcp=0,dl_dst=fa:16:3e:68:e8:19,nw_frag=no,icmp_type=0x86/0xff
Datapath actions: pop_vlan,ct(zone=1),recirc(0xc30495)
===============================================================================
recirc(0xc30495) - resume conntrack with default ct_state=trk|new (use
--ct-next to customize)
===============================================================================
Flow:
recirc_id=0xc30495,ct_state=new|trk,ct_zone=1,eth,icmp6,reg5=0x9e,reg6=0x1,in_port=1,vlan_tci=0x0000,dl_src=26:28:b0:96:c0:c7,dl_dst=fa:16:3e:68:e8:19,ipv6_src=fe80::2428:b0ff:fe96:c0c7,ipv6_dst=fe80::f816:3eff:fe68:e819,ipv6_label=0x7cc6b,nw_tos=0,nw_ecn=0,nw_ttl=255,icmp_type=134,icmp_code=0
bridge("br-int")
----------------
thaw
Resuming from table 82
82. ct_state=+new-est,ipv6,reg5=0x9e, priority 74, cookie 0x4ef408376e507615
ct(commit,zone=NXM_NX_REG6[0..15])
drop
-> Sets the packet to an untracked state, and clears all the conntrack
fields.
output:158
resubmit(,92)
92. priority 0, cookie 0x4ef408376e507615
drop
Final flow:
recirc_id=0xc30495,eth,icmp6,reg5=0x9e,reg6=0x1,in_port=1,vlan_tci=0x0000,dl_src=26:28:b0:96:c0:c7,dl_dst=fa:16:3e:68:e8:19,ipv6_src=fe80::2428:b0ff:fe96:c0c7,ipv6_dst=fe80::f816:3eff:fe68:e819,ipv6_label=0x7cc6b,nw_tos=0,nw_ecn=0,nw_ttl=255,icmp_type=134,icmp_code=0
Megaflow:
recirc_id=0xc30495,ct_state=+new-est-rel-rpl,eth,ipv6,in_port=1,nw_frag=no
Datapath actions: ct(commit,zone=1),125
3) We've verified that it's all about OVS and the packet is discarded
by the rule.
============================================Fixing the
issue:============================================
The documentation for the OVS Firewall Driver
(https://docs.openstack.org/neutron/latest/contributor/internals/openvswitch_firewall.html#rules-example-with-explanation)
lists the rule tables for neighbor solicitation and neighbor advertisement,
but there is nothing about router advertisement. We have studied the source
code that creates the necessary rules and paid attention to the
ICMPV6_ALLOWED_INGRESS_TYPES variable that accepts the list of ICMPv6 types.
We just added the required type there and it solved the problem.
neutron/agent/firewall.py
ICMPV6_ALLOWED_INGRESS_TYPES = (n_const.ICMPV6_TYPE_MLD_QUERY,
+ n_const.ICMPV6_TYPE_RA,
n_const.ICMPV6_TYPE_NS,
n_const.ICMPV6_TYPE_NA)
It solved the issue.
We got successful trace:
# docker exec openvswitch_vswitchd ovs-appctl ofproto/trace br-ext in_port=1
fa163e68e8192628b096c0c781000cb686dd6007cc6b00683afffe800000000000002428b0fffe96c0c7fe80000000000000f8163efffe68e819860010d2408000060000000000000000030440c000278d0000003840000000002a050fc102000000000000000000000019050000000000022a050fc10000000000000000000000022a050fc100000000000000000000000301012628b096c0c707010000000007d0
Flow:
icmp6,in_port=1,dl_vlan=3254,dl_vlan_pcp=0,vlan_tci1=0x0000,dl_src=26:28:b0:96:c0:c7,dl_dst=fa:16:3e:68:e8:19,ipv6_src=fe80::2428:b0ff:fe96:c0c7,ipv6_dst=fe80::f816:3eff:fe68:e819,ipv6_label=0x7cc6b,nw_tos=0,nw_ecn=0,nw_ttl=255,icmp_type=134,icmp_code=0
bridge("br-ext")
----------------
0. priority 0, cookie 0x919d5b8cf75468f6
NORMAL
-> forwarding to learned port
bridge("br-int")
----------------
0. in_port=1,dl_vlan=3254, priority 3, cookie 0x99db92e60316827
set_field:4097->vlan_vid
goto_table:60
60. dl_vlan=1,dl_dst=fa:16:3e:68:e8:19, priority 90, cookie 0x99db92e60316827
set_field:0x9e->reg5
set_field:0x1->reg6
pop_vlan
resubmit(,81)
81. icmp6,reg5=0x9e,icmp_type=134, priority 100, cookie 0x99db92e60316827
output:158
Final flow: unchanged
Megaflow:
recirc_id=0,eth,icmp6,in_port=1,dl_vlan=3254,dl_vlan_pcp=0,dl_src=26:28:b0:96:c0:c7,dl_dst=fa:16:3e:68:e8:19,nw_frag=no,icmp_type=0x86/0xff
Datapath actions: pop_vlan,125
============================================Affected
versions:===========================================
We made tests:
- Ussuri
- Zed
- Master branch
===============================================Conclusions:==============================================
1) I think that correct work has been broken by this one commit:
https://opendev.org/openstack/neutron/commit/0dcf3d20c2e5c2592e9674e7277acce4eff98341
2) I suggest returning n_const.ICMPV6_TYPE_RA to the
ICMPV6_ALLOWED_INGRESS_TYPES variable and eliminate the duplicated
ICMPv6 RA rule from the iptables firewall driver.
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2046196/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
