Re: Failover routers with OpenBGPD and independent BGP sessions
On 2006/10/19 17:57, X Y wrote: > I can't override this with nexthop are you sure? this should work. you are setting it on the _sending_ machine and not the _receiving_ machine aren't you? looking at `bgpd -nv' may help rtr2$ bgpctl sh ip bgp x.x.0.0 flags: * = Valid, > = Selected, I = via IBGP, A = Announced origin: i = IGP, e = EGP, ? = Incomplete flags destination gateway lpref med aspath origin I x.x.0/22y.y.187.61 100 0 blah blah blah i rtr1$ sudo vi /etc/bgpd.conf (add 'set nexthop self') rtr1$ bgpctl reload rtr2$ bgpctl sh ip bgp x.x.0.0 flags: * = Valid, > = Selected, I = via IBGP, A = Announced origin: i = IGP, e = EGP, ? = Incomplete flags destination gateway lpref med aspath origin I x.x.0/22y.y.187.35 100 0 blah blah blah i > 1) Will the "set localpref -10" on the session with the other router > be sufficient to make sure that when the main BGP session is up, > that's actually used? yes, localpref overrides everything else (*including* AS path length). http://unduli.bsws.de/papers/linuxforum2006/mgp00016.txt 1. check if prefix is eligible a.k.a reachable 2. localpref, bigger is better 3. aspath length, the shorter the better 4. origin, the lower the better 5. MED decision, only comparable between the same neighboring AS 6. EBGP is cooler than IBGP 7. weight, bigger is better (extension) 8. route age: older is better (extension, off by default) 9. lowest BGP ID wins 10. lowest peer address wins most of it is standard BGP but the extensions aren't, this list should probably be added to bgpd(8) or possibly bgp.conf(5)... (if anyone would like to express a preference as to which manpage I can prepare a diff) > 2) When I get to use multiple locations, should I use ospfd rather > than BGP to manage which route to take internally to the network? I assume you are talking about using it to tell which route to take from the border to your internal networks (hosted machines and so on)? that's up to you :-)
Re: Failover routers with OpenBGPD and independent BGP sessions
X Y wrote : I'm having a bit of trouble with the finer details of my OpenBGPD config, and would appreciate some tips on getting it right and advice on the right way of doing things. I have two routers, two independent BGP connections, and a block of provider independent address space. The routers are arranged in a redundant pair. The public network and some private subnets have gateway addresses provided with CARP. The two routers use pfsync. The BGP connections are actually completely independent (I'll be adding two more in due course for a total of four). They have different network addresses, cables and route to the rest of the world. The cables are plugged directly into the routers, and there's no CARP on those interfaces. Packets will arrive via either of those routes. I have got a basic configuration working. This maintains the BGP sessions, packets go in and out, and the firewalls will fail over as they should. I use depend on carp0 ... carp3 on the master router (chosen via advskew) to drop that session if it fails, and demote on the backup to make sure it doesn't like being master if it doesn't have a BGP session. I have been recommended by our ISPs that I should also advertise routes between the routers, so that if one's BGP session fails, it can route packets to the other for a cleaner failover. I have not managed to get this configuration working. Some configuration information, with the real details removed to protect the guilty. AS: 9 PI subnet: A.A.A.0/23 PI gateway: A.A.A.1 Master: A.A.A.2 Backup: A.A.A.3 BGP connection 1: X.X.X.4 -> X.X.X.200 on X.X.X.0/24, AS 8 BGP connection 2: Y.Y.Y.4 -> Y.Y.Y.200 on Y.Y.Y.0/24, AS 8 (Y.Y.Y !=3D X.X.X) /etc/bgpd.conf AS 9 network A.A.A.0/23 neighbor X.X.X.200 { remote-as 8 local-address X.X.X.4 announce self tcp md5sig password PASSWORD1 depend on carp1 depend on carp2 depend on carp3 # demote on backup } neighbor A.A.A.3 { remote-as 9 descr "backup" local-address A.A.A.2 announce all tcp md5sig password PASSWORD2 set nexthop A.A.A.3 # A.A.A.2 didn't help set localpref -10 } Then... Stuart Henderson <[EMAIL PROTECTED]> wrote: On 2006/10/13 11:24, Ronnie Garcia wrote: I have been recommended by our ISPs that I should also advertise routes between the routers, so that if one's BGP session fails, =20 it can route packets to the other for a cleaner failover. I have not =20 managed to get this configuration working. Yes you should, this is called iBGP. All of your BGP routers =20 should have a iBGP session with all of the others, in a full mesh (unless you are using a route reflector). OP has already done that in the config file, the problem is how to add a route so the other provider's router can be reached. Normally =20= the provider's router is listed in the IBGP announcement so unless this is overwritten in the IBGP announcements (by 'set nexthop) you need to =20= have a route to the provider's router (static or OSPF). I think this is the critical bit of information. I need to add a =20 static route to the other router for the X.X.X/24 or Y.Y=10.Y/24 =20 network. The IBGP session from the other router will give it's =20 neighbour's address, not it's own address which I had expected. I =20 can't override this with nexthop, I just have to make sure there are =20 routes provided via something other than BGP. A couple of follow-up questions: 1) Will the "set localpref -10" on the session with the other router =20 be sufficient to make sure that when the main BGP session is up, =20 that's actually used? 2) When I get to use multiple locations, should I use ospfd rather =20 than BGP to manage which route to take internally to the network? Thanks for the help, Ben
Re: Failover routers with OpenBGPD and independent BGP sessions
To me it seems that even having the IBGP session won't help the OP's particular issue (though he should have it anyway for other reasons)... as the peer session goes down, the routes from it go down with it, and IBGP withdraws those announcements. Nothing gets held over. Maybe establishing a second peering session with each provider will help in your redundancy- that's what I currently do, and yes, it has come in handy. I'm one week away from implementing OpenBGPD as route servers for maximum BGP redundancy. Here's my plan, maybe you can take something away from it for your own solution for redundancy- I have multiple egress points in my network (separate physical datacenters (DC's) with their own carriers, with the DC's linked together via Layer2 fiber access.) I use OpenBGPD as a route server (RS) at each DC to feed my edge routers a custom single table via IBGP (meaning the RS's are making the real EBGP multihop peerings with my carriers), making the edge routers essentially just packet forwarders. The DC's have layer2 access to each other, so I will populate each DC with an RS with CARP (in a VLAN) so the edge routers (and the carriers) only see one RS at a time, but multiple RS's are on standby. If a DC, edge router, RS, carrier, or Layer2 connections between the DC's fail, the RS's will be able to recover and continue to peer- either with everyone, or just their closest edge router and carriers (each DC has its own separate address blocks, so a split situation won't be a conflict.) As an extra step of redundancy against the RS's, I can have those edge routers form EBGP sessions with their directly connected peers, but only announcing a basic set of our prefixes (not individual /24's like the OpenBGPD RS's do for better traffic engineering, and only accepting a default route.) That way, if the entire redundant OpenBGPD router server model fails (which I doubt will happen), I still have basic routing with my carriers. Dan Farrell > -Original Message- > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of > Stuart Henderson > Sent: Friday, October 13, 2006 6:45 AM > To: misc@openbsd.org > Subject: Re: Failover routers with OpenBGPD and independent BGP sessions > > On 2006/10/13 11:24, Ronnie Garcia wrote: > > >I have been recommended by our ISPs that I should also advertise > > >routes between the routers, so that if one's BGP session fails, it can > > >route packets to the other for a cleaner failover. I have not managed > > >to get this configuration working. > > > > Yes you should, this is called iBGP. All of your BGP routers should have > > a iBGP session with all of the others, in a full mesh (unless you are > > using a route reflector). > > OP has already done that in the config file, the problem is how to > add a route so the other provider's router can be reached. Normally the > provider's router is listed in the IBGP announcement so unless this is > overwritten in the IBGP announcements (by 'set nexthop) you need to have > a route to the provider's router (static or OSPF). > > From the sample config I guess OP may not realise that 'set nexthop' is > on announcements, it doesn't overwrite the nexthop on incoming routes. > Any confusion with the action of 'set' attributes, use bgpd -nv which > demonstrates clearly how they apply. > > On 2006/10/12 14:05, X Y wrote: > > I use depend on carp0 ... carp3 on the master router > > (chosen via advskew) to drop that session if it fails, > > 'depend on carp' is mostly for where you have a peer session running > from a carp address (e.g. if you're at an IXP where you are only > allowed one IP address and want to connect two routers). It is used > to hold the backup router in IDLE in normal conditions, and > immediately connect when it becomes master. I'm not sure this is > what you are intending to do? > > It might help to see some 'sh ip bgp d '.
Re: Failover routers with OpenBGPD and independent BGP sessions
On 2006/10/13 11:24, Ronnie Garcia wrote: > >I have been recommended by our ISPs that I should also advertise > >routes between the routers, so that if one's BGP session fails, it can > >route packets to the other for a cleaner failover. I have not managed > >to get this configuration working. > > Yes you should, this is called iBGP. All of your BGP routers should have > a iBGP session with all of the others, in a full mesh (unless you are > using a route reflector). OP has already done that in the config file, the problem is how to add a route so the other provider's router can be reached. Normally the provider's router is listed in the IBGP announcement so unless this is overwritten in the IBGP announcements (by 'set nexthop) you need to have a route to the provider's router (static or OSPF). >From the sample config I guess OP may not realise that 'set nexthop' is on announcements, it doesn't overwrite the nexthop on incoming routes. Any confusion with the action of 'set' attributes, use bgpd -nv which demonstrates clearly how they apply. On 2006/10/12 14:05, X Y wrote: > I use depend on carp0 ... carp3 on the master router > (chosen via advskew) to drop that session if it fails, 'depend on carp' is mostly for where you have a peer session running from a carp address (e.g. if you're at an IXP where you are only allowed one IP address and want to connect two routers). It is used to hold the backup router in IDLE in normal conditions, and immediately connect when it becomes master. I'm not sure this is what you are intending to do? It might help to see some 'sh ip bgp d '.
Re: Failover routers with OpenBGPD and independent BGP sessions
X Y a icrit : I have two routers, two independent BGP connections, and a block of provider independent address space. The routers are arranged in a redundant pair. The public network and some private subnets have gateway addresses provided with CARP. The two routers use pfsync. The BGP connections are actually completely independent (I'll be adding two more in due course for a total of four). They have different network addresses, cables and route to the rest of the world. The cables are plugged directly into the routers, and there's no CARP on those interfaces. Packets will arrive via either of those routes. I have got a basic configuration working. This maintains the BGP sessions, packets go in and out, and the firewalls will fail over as they should. I use depend on carp0 ... carp3 on the master router (chosen via advskew) to drop that session if it fails, and demote on the backup to make sure it doesn't like being master if it doesn't have a BGP session. I have been recommended by our ISPs that I should also advertise routes between the routers, so that if one's BGP session fails, it can route packets to the other for a cleaner failover. I have not managed to get this configuration working. Yes you should, this is called iBGP. All of your BGP routers should have a iBGP session with all of the others, in a full mesh (unless you are using a route reflector). In your design, you will then get the best routes on each of your border routers. Some configuration information, with the real details removed to protect the guilty. AS: 9 PI subnet: A.A.A.0/23 PI gateway: A.A.A.1 Master: A.A.A.2 Backup: A.A.A.3 BGP connection 1: X.X.X.4 -> X.X.X.200 on X.X.X.0/24, AS 8 BGP connection 2: Y.Y.Y.4 -> Y.Y.Y.200 on Y.Y.Y.0/24, AS 8 (Y.Y.Y != X.X.X) [...] neighbor A.A.A.3 { remote-as 9 descr "backup" local-address A.A.A.2 announce all tcp md5sig password PASSWORD2 set nexthop A.A.A.3# A.A.A.2 didn't help set localpref -10 } You shouldn't need a nexthop here. In iBGP sessions, you should set the neighbor address to be the loopback address of your other border router. Your router-id parameter should also be the IP adress of your local loopback interface. Your loopback interfaces should have a /32 IP adress set. Regards, -- Ronnie Garcia