Hi all,
I'm currently trying to adopt OVN native BGP/EVPN support to our cloud'
usecases and faced one problem:
Imagine topology of one "external" VTEP node and 2 OVN-managed hosts:
one of them is a gateway chassis, and another is just a normal workload
chassis (without EVPN connectivity).
XXXXXXXXXXXXXXX
X X
XXXXXXXXXXXXXXXXX IP fabric XXXXXXXXXXXXXXX
X X X X
X XXXXXXXXXXXXXXX X
X X X
X X X
XXXXXXXXXXXXXX XXXXXXXXXXXX XXXXXXXXXXXX
X X X X X X
X Ext VTEP X X OVN GW X X OVN w/l X
X X X X X X
XXXXXXXXXXXXXX XXXXXXXXXXXX XXXXXXXXXXXX
10.128.0.5 10.128.0.14 10.128.0.6
VTEP node has configured VRF linked to VNI 300 and announces Type-5
route with 172.31.0.0/24 IP prefix.
[root@vtep01 ~]# ip a s br-300
19: br-300: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue
master ovnvrf300 state UP group default qlen 1000
link/ether 00:04:00:00:00:01 brd ff:ff:ff:ff:ff:ff
inet 172.31.0.1/24 scope global br-300
valid_lft forever preferred_lft forever
[root@vtep01 ~]# ip a s ovnvrf300
18: ovnvrf300: <NOARP,MASTER,UP,LOWER_UP> mtu 65575 qdisc noqueue state
UP group default qlen 1000
link/ether c2:9d:fb:b7:74:6f brd ff:ff:ff:ff:ff:ff
[root@vtep01 ~]# ip a s vxlan300
23: vxlan300: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue
master br-300 state UNKNOWN group default qlen 1000
link/ether 2a:39:b8:a7:b6:80 brd ff:ff:ff:ff:ff:ff
inet6 fe80::2839:b8ff:fea7:b680/64 scope link
valid_lft forever preferred_lft forever
OVN has EVPN-configured edge LR and localnet-enabled LS with static
route "10.1.0.0/24" with IPv6 LLA nexthop and output LRP which leads to LR1.
LR1 has two LRPs: one to edge LR, another is a link to "workloads".
"workload" LS, which has 2 LSPs: one on gateway node, another on the
"workload" node.
OVN topology is as next:
switch b534bd4c-02bb-4428-a834-fbb21b59d79f (ls1)
port ls1-lr1
type: router
router-port: lr1-downlink
port lsp2
addresses: ["00:00:00:00:00:02 10.1.0.102"]
port lsp1
addresses: ["00:00:00:00:00:01 10.1.0.101"]
switch f03fa0e5-46a9-49b8-91b7-2e506977170c (test-ovn1-uplink)
port ln-node2
type: router
router-port: edge1-uplink2
port test-ovn1-uplink-lsp
type: localnet
addresses: ["unknown"]
port test-ovn1-edge1
type: router
router-port: edge1-uplink
router d322ecc0-c225-49f0-aa37-7c3003097714 (edge1)
port edge1-lr1
mac: "00:02:00:00:00:01"
ipv6-lla: "fe80::202:ff:fe00:1"
port edge1-uplink
mac: "0a:00:aa:21:e5:e0"
ipv6-lla: "fe80::800:aaff:fe21:e5e0"
networks: ["10.128.0.101/20"] <--- I'm not sure it is correct
gateway chassis: [8fdef43f-1bdb-4806-b2bd-bab918ee98f9]
router fab999fe-6ef7-478e-9b42-494cafe8f4b1 (lr1)
port lr1-downlink
mac: "00:03:00:00:00:01"
ipv6-lla: "fe80::203:ff:fe00:1"
networks: ["10.1.0.1/24"]
port lr1-edge1
mac: "00:02:00:00:00:02"
ipv6-lla: "fe80::202:ff:fe00:2"
# ovn-nbctl lr-route-list edge1
IPv4 Routes
Route Table <main>:
10.1.0.0/16 fe80::202:ff:fe00:2 dst-ip edge1-lr1
# ovn-nbctl lr-route-list lr1
IPv4 Routes
Route Table <main>:
0.0.0.0/0 fe80::202:ff:fe00:1 dst-ip lr1-edge1
# ovn-nbctl get logical-switch test-ovn1-uplink other_config
{dynamic-routing-advertise-ifname=lo-300,
dynamic-routing-bridge-ifname=br-300,
dynamic-routing-redistribute="fdb,ip", dynamic-routing-vni="300",
dynamic-routing-vxlan-ifname=vxlan300}
# ovn-nbctl get logical-router edge1 options
{dynamic-routing="true", dynamic-routing-redistribute=static,
dynamic-routing-vrf-id="300"}
# ovn-nbctl get logical-router-port edge1-uplink options
{dynamic-routing-maintain-vrf="true",
dynamic-routing-redistribute=connected}
So, if I start pinging from VTEP's VRF (from 172.31.0.0/24) to LSP1
(10.1.0.101), which is on the gateway node, it works fine.
On the OVN GW node the evpn commands have next output:
# ovn-appctl evpn/remote-vtep-list
IP: 10.128.0.5, port: 4789, vni: 300
# ovn-appctl evpn/vtep-arp-list
UUID: 7e3f9194-0699-4ed2-83c3-73b474c1b2b1, VNI: 300, MAC:
00:04:00:00:00:01, IP: 10.128.0.5, dp_key: 5
# ovn-appctl evpn/vtep-binding-list
UUID: 826cd9a5-d582-4a3a-85d9-2f6471c0afa6, Remote IP: 10.128.0.5, vni:
300, binding_key: 0x80000015, tunnel_ofport: 10, dp_key: 5
# ovn-appctl evpn/vtep-fdb-list
UUID: c44b7556-e8f9-4bce-9d8b-9c916b79d105, MAC: 00:04:00:00:00:01, vni:
300, binding_key: 0x80000015, dp_key: 5
# ovn-appctl evpn/vtep-multicast-group-list
UUID: 81f25768-6a40-497e-b48f-e17faa91e0e7, Remote IPs: 10.128.0.5, vni: 300
PROBLEM:
BUT, if I start pinging from VTEP node to LSP2 (10.1.0.102), it fails.
ICMP request goes from VTEP node to OVN GW node via VXLAN VNI300 (fine),
then OVN GW sends this traffic inside GENEVE to OVN w/l node and ICMP
request is seen on the VIF (lsp2).
ICMP response is sent from lsp2, but I see ARP request in the GENEVE -
OVN w/l node doesn't know br-300 MAC address on the VTEP node (which is
the dst MAC for ICMP response).
On then one hand that is expected - EVPN mac addresses are stored in the
ovn-controller runtime and not committed to OVN SB, so OVN w/l node
doesn't know which mac to use.
On the other hand, maybe I've lost some configuration hints and such
topology can work as well?
P.S. in ipv6 scenario, if I don't set 'networks' on the gateway LRP,
ovn-northd generated incorrect logical flow (example):
# ovn-sbctl dump-flows edge1 | grep 'null'
table=16(lr_in_ip_routing ), priority=514 , match=(reg7 == 0 &&
ip6.dst == 2002::/64), action=(ip.ttl--; reg8[0..15] = 0; reg0 =
10.128.0.5; reg5 = (null); eth.src = 0a:00:aa:21:e5:e0; outport =
"edge1-uplink"; flags.loopback = 1; reg9[9] = 1; next;)
So, I'm very concerned about which IP addresses should be used in GW LRP
'networks'? Why do we need them at all?
--
regards,
Vladislav Odintsov
_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
