Hi,
So we are currently trying to set up one mgre interface instead of multiple gre
tunnel between two vpn machines and we are running against a problem with bgpd.
we have two machines, vpn1 and vpn2, we have set up an mgre interface on both
like this:
root@vpn1:~ # ifconfig mgre0
mgre0: flags=8841<UP,RUNNING,SIMPLEX,MULTICAST> mtu 1476
index 15 priority 0 llprio 3
encap: vnetid none
groups: mgre
tunnel: inet <public ip1> ttl 64 nodf
inet 172.29.1.2 netmask 0xffffff00
root@vpn1:~ # route -n show | grep 172.29.1
172.29.1/24 172.29.1.3 UCn 0 0 - 4 mgre0
172.29.1.2 <public ip2> UHS 1 21 - L 8 mgre0
172.29.1.3 mgre0 UHl 0 309 - 1 mgre0
root@vpn2:~ # ifconfig mgre0
mgre0: flags=8841<UP,RUNNING,SIMPLEX,MULTICAST> mtu 1476
index 15 priority 0 llprio 3
encap: vnetid none
groups: mgre
tunnel: inet 192.168.0.3 ttl 64 nodf
inet 172.29.1.3 netmask 0xffffff00
root@vpn2:~ # route -n show | grep 172.29.1
172.29.1/24 172.29.1.2 UCn 0 0 - 4 mgre0
172.29.1.2 mgre0 UHl 0 1295 - 1 mgre0
172.29.1.3 <public ip1> UHS 1 39 - L 8 mgre0
The tunnel is up and reachable:
root@vpn1:~ # ping -I 172.29.1.2 172.29.1.3
PING 172.29.1.3 (172.29.1.3): 56 data bytes
64 bytes from 172.29.1.3: icmp_seq=0 ttl=255 time=12.351 m
We then have a bgp session up as follow:
neighbor 172.29.1.3 {
descr "vpn1"
local-address 172.29.1.2
remote-as 64660
announce IPv4 unicast
announce IPv6 none
holdtime 25
announce all
}
Bgp tunnel is up:
root@vpn1:~ # bgpctl show
Neighbor AS MsgRcvd MsgSent OutQ Up/Down State/PrfRcvd
vpn1 64660 329 201 0 00:17:10 410
the bgp fib table shows the prefix received properly:
root@vpn1:~ # bgpctl show fib
flags: * = valid, B = BGP, C = Connected, S = Static, D = Dynamic
N = BGP Nexthop reachable via this route R = redistributed
r = reject route, b = blackhole route
flags prio destination gateway
*B 48 10.1.0.0/24 <public ip1>
*B 48 10.1.2.0/24 <public ip1>
*B 48 10.1.3.0/24 <public ip1>
*B 48 10.1.4.0/24 <public ip1>
*B 48 10.1.5.0/24 <public ip1>
*B 48 10.1.6.0/24 <public ip1>
*B 48 10.1.16.0/24 <public ip1>
*B 48 10.1.18.0/24 <public ip1>
*B 48 10.1.19.0/24 <public ip1>
*B 48 10.1.20.0/24 <public ip1>
*B 48 10.1.21.0/24 <public ip1>
... snip
and rib table:
root@vpn1:~ # bgpctl show rib
flags: * = Valid, > = Selected, I = via IBGP, A = Announced, S = Stale
origin: i = IGP, e = EGP, ? = Incomplete
flags destination gateway lpref med aspath origin
*> 10.1.0.0/24 172.29.1.3 100 1003000 64660 64901 64740 i
*> 10.1.2.0/24 172.29.1.3 100 1361100 64660 64901 i
*> 10.1.3.0/24 172.29.1.3 100 2000100 64660 64901 i
*> 10.1.4.0/24 172.29.1.3 100 1010300 64660 64901 64710 i
*> 10.1.5.0/24 172.29.1.3 100 1365100 64660 64901 64711 i
*> 10.1.6.0/24 172.29.1.3 100 1001200 64660 64901 64712 i
*> 10.1.16.0/24 172.29.1.3 100 1003000 64660 64901 64740 i
*> 10.1.18.0/24 172.29.1.3 100 1361100 64660 64901 i
*> 10.1.19.0/24 172.29.1.3 100 2000100 64660 64901 i
*> 10.1.20.0/24 172.29.1.3 100 1010300 64660 64901 64710 i
*> 10.1.21.0/24 172.29.1.3 100 1365100 64660 64901 64711 i
root@vpn1:~ # bgpctl show fib next
flags: * = valid, B = BGP, C = Connected, S = Static, D = Dynamic
N = BGP Nexthop reachable via this route R = redistributed
r = reject route, b = blackhole route
flags prio destination gateway
*SNR 8 172.29.1.3/32 <public ip1>
root@vpn1:~ # bgpctl show next
Flags: * = nexthop valid
Nexthop Route Prio Gateway Iface
* 172.29.1.3 172.29.1.3/32 8 <public ip1> mgre0 (UP, unknown)
But we can't see those prefixes added to the kernel routing table:
r...@vpn1.atc.kambi.com(master):~ # route -n show | grep "10\.1\."
r...@vpn1.atc.kambi.com(master):~ # route -n show | grep mgre
172.29.1/24 172.29.1.2 UCn 0 0 - 4 mgre0
172.29.1.2 mgre0 UHl 0 1445 - 1 mgre0
172.29.1.3 <public ip1> UHS 1 40 - L 8 mgre0
We don't see what we are missing here, of if it's a bug.
What else can we troubleshoot to make mgre work with bgpd?
Thanks,
Ben
