Le 21 nov. 2018 à 20:26, David Ahern <d...@cumulusnetworks.com> a écrit : > > On 11/21/18 6:30 AM, Alexis Bauvin wrote: >> Le 20 nov. 2018 à 22:45, David Ahern <d...@cumulusnetworks.com> a écrit : >>> >>> On 11/20/18 7:23 AM, Alexis Bauvin wrote: >>>> We are trying to isolate the VXLAN traffic from different VMs with VRF as >>>> shown >>>> in the schemas below: >>>> >>>> +-------------------------+ +----------------------------+ >>>> | +----------+ | | +------------+ | >>>> | | | | | | | | >>>> | | tap-red | | | | tap-blue | | >>>> | | | | | | | | >>>> | +----+-----+ | | +-----+------+ | >>>> | | | | | | >>>> | | | | | | >>>> | +----+---+ | | +----+----+ | >>>> | | | | | | | | >>>> | | br-red | | | | br-blue | | >>>> | | | | | | | | >>>> | +----+---+ | | +----+----+ | >>>> | | | | | | >>>> | | | | | | >>>> | | | | | | >>>> | +----+--------+ | | +--------------+ | >>>> | | | | | | | | >>>> | | vxlan-red | | | | vxlan-blue | | >>>> | | | | | | | | >>>> | +------+------+ | | +-------+------+ | >>>> | | | | | | >>>> | | VRF | | | VRF | >>>> | | red | | | blue | >>>> +-------------------------+ +----------------------------+ >>> >>> Roopa and I were discussing this setup and are puzzled by the VRF >>> association here. Does br-red and br-blue have an address? The commands >>> below do not show it and from our perspective seems odd for this >>> scenario. If it does not have an address, then there is no reason for >>> the VRF labeling. >> >> Yes, br-red and br-blue have an address. Addresses (both MAC and IP) >> are anycast addresses, to make an anycast gateway for the VMs behind >> the taps. This serves for a classical evpn symmetric distributed >> routing, both between vxlans and towards external networks. I am using >> FRR as a control plane to exchange bgp-evpn information between >> hypervisors. >> >> The schematic could have been more complete, including more bridges >> and more taps in the vrf, with an added bridge+vxlan for the l3vni. >> I did not include them as I wanted to focus on the underlay itself >> and not on what vxlan was used for. >> >> Here is a more complete example: >> +---------+ >> | | >> | vrf1000 | >> | | >> +--+-+-+--+ >> | | | >> +---------------------+ | +---------------+ >> | | | >> +------+------+ +------+------+ +----+---+ >> | | | | | | >> | br0 | | br1 | | br1000 | >> | 10.0.0.1/24 | | 10.0.1.1/24 | | | >> | | | | +----+---+ >> +----+-+-+----+ +-----+-+-----+ | >> | | | | | | >> +-------+ | +-------+ +--+ +---+ | >> | | | | | | >> +--+---+ +---+--+ +----+---+ +---+--+ +---+----+ +-----+-----+ >> | | | | | | | | | | | | >> | tap0 | | tap1 | | vxlan0 | | tap2 | | vxlan1 | | vxlan1000 | >> | | | | | | | | | | | | >> +------+ +------+ +--------+ +------+ +--------+ +-----------+ >> . . . >> . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . >> . >> . +---------+ +--------------+ >> . | | | | >> . | vrf1001 | | vrf-underlay | >> . | | | | >> . +---+-+---+ +-------+------+ >> . | | | >> . +---------+ +----------+ | >> . | | | >> . +------+------+ +----+---+ | >> . | | | | | >> . | br2 | | br1001 | | >> . | 10.0.2.1/24 | | | | >> . | | +----+---+ | >> . +----+-+-+----+ | | >> . | | | | | >> . +-----+ | +-------+ | | >> . | | | | +-------+-------+ >> . +---+--+ +--+---+ +---+----+ +-----+-----+ | | >> . | | | | | | | | | eth0.2030 | >> . | tap3 | | tap4 | | vxlan2 | | vxlan1001 | | 172.16.0.2/24 | >> . | | | | | | | | | | >> . +------+ +------+ +--------+ +-----------+ +---------------+ >> . . . ^ . >> . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . >> (vxlan lower device is eth0.2030) v >> +----------------+ >> | | >> | eth0 | >> | 192.168.1.2/24 | >> | | >> +----------------+ >> >> In this example, the underlay is rather simple. A more complex case >> could be with several uplinks in the underlay vrf, and a dummy/loopback >> with a /32 ip, used as a vxlan lower device and to bind routing daemons >> (e.g. bgpd). >> >> +--------------+ >> | | >> | vrf-underlay | >> | | >> +-----+-+-+----+ >> | | | >> +---------------+ | +---------------+ >> | | | >> +-------+-------+ +-------+-------+ +-------+--------+ >> | | | | | | >> | eth1 | | eth2 | | dummy0 | >> | 172.16.0.2/31 | | 172.16.0.4/31 | | 192.168.0.1/32 | >> | | | | | | >> +---------------+ +---------------+ +----------------+ >> . >> . . . . . . . . . . . . . . . . . . . . . . . >> (vxlan lower device is dummy0) > > Thanks for sending a more complete example. I forwarded it to some FRR > folks as well.
My pleasure. >> >> As for default vrf, it would contain for example management traffic. We >> don’t want to mix the underlay routing tables to other traffics. > > Management VRF? :-) Definitely possible, indeed. This would be the best solution, if only I *could*. Reality is, people are still suspicious of networking guys. The least we touch their servers, the better they are. For hypervisors we can, but the rest of them are a mix and match, and I need to be as generic as possible. Telling others to get off the default vrf for their own management softwares (for example) if they want to integrate into what I’m developing is quite the challenge. I need to be as plug-and-play as possible, and an underlay vrf is the best option to keep the default to whatever it was used for. (And as weird as it might be, it is easier for me to tell them to upgrade to a mainline kernel than to rewrite their networking setup) >> >>> Also, it would be good to have a unit test this case. Can you create a >>> shell script that creates the setup and runs a few tests verifying >>> connectivity? You can use network namespaces and veth pairs in place of >>> the VM with a tap device. From there the functionality is the same. >>> Tests can be initial VRF association for the vxlan lower device, >>> changing the VRF to another device, and then changing again back to >>> default VRF - checking proper connectivity for each. >> >> Sure! I’ve just finished writing it, but I am not sure of the best way >> so send it. I guess I will have to add it to a v4 of the patches, in >> tools/testing/selftests/net? > > yes, that would be good. Thank you for writing it. Will be sent in next patch series.
