bgpd does not re-route correctly when I shut down a transit when I use a bgp-only design, causing black-holes for some prefixes.
router-01 and router-02 are in the same AS and peer with the same transit provider. router-01 and router-02 have two ibgp peerings, primary and standby path. router-01 sets localpref 60 on all transit prefixes, router-02 sets local-pref 50. When I take down the transit on router-01 I see this on router-02: router-02# bgpctl show rib | head -n 10 flags: * = Valid, > = Selected, I = via IBGP, A = Announced origin: i = IGP, e = EGP, ? = Incomplete flags destination gateway lpref med aspath origin I*> 26.0.128.0/17 172.17.1.1 60 11100 65100 i * 26.0.128.0/17 192.168.100.5 50 10100 65100 i I*> 26.0.144.0/22 172.17.1.1 60 11100 65100 i * 26.0.144.0/22 192.168.100.5 50 10100 65100 i I*> 26.1.77.0/24 172.17.1.1 60 11100 65100 i * 26.1.77.0/24 192.168.100.5 50 10100 65100 i router-02# prefixes with local-pref 60 pointing at router-01. router-01 does not have it's transit peering up, and thus itself has no prefixes with local-pref 60. router-01# bgpctl show rib | head -n 10 flags: * = Valid, > = Selected, I = via IBGP, A = Announced origin: i = IGP, e = EGP, ? = Incomplete flags destination gateway lpref med aspath origin I*> 26.0.128.0/17 172.17.1.6 50 21100 65100 i I*> 26.0.144.0/22 172.17.1.6 50 21100 65100 i I*> 26.1.77.0/24 172.17.1.6 50 21100 65100 i I*> 26.2.172.0/22 172.17.1.6 50 21100 65100 i I*> 26.3.241.0/24 172.17.1.6 50 21100 65100 i I*> 26.6.126.0/24 172.17.1.6 50 21100 65100 i router-01# bgpctl show rib 26.0.128.0/17 all flags: * = Valid, > = Selected, I = via IBGP, A = Announced origin: i = IGP, e = EGP, ? = Incomplete flags destination gateway lpref med aspath origin I*> 26.0.128.0/17 172.17.1.6 50 21100 65100 i I*> 26.0.144.0/22 172.17.1.6 50 21100 65100 i router-01# I saw this before when I tested bgpd around a year ago. So it isn't a new bug. This is with 4.2-RELEASE, no patches. This info is from a lab I setup to replicate a live environment. /Tony router-01# cat /etc/bgpd.conf # $OpenBSD: bgpd.conf,v 1.8 2007/03/29 13:37:35 claudio Exp $ # sample bgpd configuration file # see bgpd.conf(5) #macros loopback="172.17.0.1" # global configuration AS 65200 router-id $loopback network $loopback/32 set {localpref 120, med 10} network 172.17.0.0/16 set {localpref 120, med 10} network connected set {localpref 120, med 10} network static set {localpref 120, med 10} group "TRANSIT" { remote-as 65100 announce all set nexthop self set med 10100 set localpref 60 neighbor 192.168.100.1 { descr "TRANSIT" } } group "IBGP" { remote-as 65200 route-reflector set nexthop self set med +1000 neighbor 172.17.1.2 { local-address 172.17.1.1 descr "router-02 primary" } neighbor 172.17.1.6 { local-address 172.17.1.5 descr "router-02 standby" set med +10000 } } # filter deny from any deny to any allow quick to group "IBGP" allow quick from group "IBGP" allow quick to group "TRANSIT" prefix 172.17.0.0/16 allow quick from group "TRANSIT" router-01# ifconfig lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 33208 groups: lo inet 127.0.0.1 netmask 0xff000000 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5 ne3: flags=8863<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST> mtu 1500 lladdr 52:54:00:12:02:01 description: transit media: Ethernet 10baseT full-duplex inet6 fe80::5054:ff:fe12:201%ne3 prefixlen 64 scopeid 0x1 inet 192.168.100.2 netmask 0xfffffffc broadcast 192.168.100.3 ne4: flags=8863<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST> mtu 1500 lladdr 52:54:00:12:02:02 description: router-01 primary path media: Ethernet 10baseT full-duplex inet6 fe80::5054:ff:fe12:202%ne4 prefixlen 64 scopeid 0x2 inet 172.17.1.1 netmask 0xfffffffc broadcast 172.17.1.3 ne5: flags=8863<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST> mtu 1500 lladdr 52:54:00:12:02:03 description: route-02 standby path media: Ethernet 10baseT full-duplex inet6 fe80::5054:ff:fe12:203%ne5 prefixlen 64 scopeid 0x3 inet 172.17.1.5 netmask 0xfffffffc broadcast 172.17.1.7 enc0: flags=0<> mtu 1536 lo1: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 33208 description: ROUTING LOOPBACK groups: lo inet 172.17.0.1 netmask 0xffffffff router-01# router-02# cat /etc/bgpd.conf # $OpenBSD: bgpd.conf,v 1.8 2007/03/29 13:37:35 claudio Exp $ # sample bgpd configuration file # see bgpd.conf(5) #macros loopback="172.17.0.2" # global configuration AS 65200 router-id $loopback network $loopback/32 set {localpref 120, med 10} network 172.17.0.0/16 set {localpref 120, med 10} network connected set {localpref 120, med 10} network static set {localpref 120, med 10} group "TRANSIT" { remote-as 65100 announce all set nexthop self set med 10100 set localpref 50 neighbor 192.168.100.5 { descr "TRANSIT" } } group "IBGP" { remote-as 65200 route-reflector set nexthop self set med +1000 neighbor 172.17.1.1 { local-address 172.17.1.2 descr "router-01 primary" } neighbor 172.17.1.5 { local-address 172.17.1.6 descr "router-01 standby" set med +10000 } } # filter deny from any deny to any allow quick to group "IBGP" allow quick from group "IBGP" allow quick to group "TRANSIT" prefix 172.17.0.0/16 allow quick from group "TRANSIT" router-02# ifconfig lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 33208 groups: lo inet 127.0.0.1 netmask 0xff000000 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5 ne3: flags=8863<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST> mtu 1500 lladdr 52:54:00:12:03:01 description: transit media: Ethernet 10baseT full-duplex inet6 fe80::5054:ff:fe12:301%ne3 prefixlen 64 scopeid 0x1 inet 192.168.100.6 netmask 0xfffffffc broadcast 192.168.100.7 ne4: flags=8863<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST> mtu 1500 lladdr 52:54:00:12:03:02 description: router-02 primary path media: Ethernet 10baseT full-duplex inet6 fe80::5054:ff:fe12:302%ne4 prefixlen 64 scopeid 0x2 inet 172.17.1.2 netmask 0xfffffffc broadcast 172.17.1.3 ne5: flags=8863<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST> mtu 1500 lladdr 52:54:00:12:03:03 description: router-02 standby path media: Ethernet 10baseT full-duplex inet6 fe80::5054:ff:fe12:303%ne5 prefixlen 64 scopeid 0x3 inet 172.17.1.6 netmask 0xfffffffc broadcast 172.17.1.7 enc0: flags=0<> mtu 1536 lo1: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 33208 groups: lo inet 172.17.0.2 netmask 0xffffffff router-02#