Re: carp, ospf can't see carp state
On Mon, Apr 09, 2007 at 02:03:21PM -0400, Frangois Rousseau wrote: > Hi Claudio, > > I have double check on my lab and everything work fine for the OSPF > part, sorry for my mistake. > > But at the end, I'm still having the same problem: the server didn't > know the right route. > > OSPF see all the route correctly but the system didn't seem to be > updated. If I do "route show" I only see the local route pointing > directly to the CARP device instead of pointing to the other router. > > route show give me something like this when my cable is unplug from > the "carp" interface: > 83.201.77/24link#10UC 0 0 - carp1 > > What do you think it can be? > ospfd will never overwrite already present routes (unless they have came from bgpd). So the carp route can not be changed. AFAIK you only get such a network if you are using a unnumbered parent device. Could you try to give the parent interface an IP address out of 83.201.77/24 -- this should change the link local route to this network to the real interface. This will solve the problem in case the box is BACKUP. There is still a problem when you unplug the network. In this case packets hitting that box will get dropped. This can only be fixed if the kernel is able to change the RTF_UP flag depending on the link state. -- :wq Claudio > > > 2007/4/7, Claudio Jeker <[EMAIL PROTECTED]>: > >On Sat, Apr 07, 2007 at 12:21:19PM -0400, Frangois Rousseau wrote: > >> But how I'm suppose to annonce the route for the right carp interface? > >> Right now my servers can always reach the router because of the CARP > >> interface but the router can't always reach the servers... > >> > >> If I unplug the cable of my CARP interface (bge2 for example), all > >> traffic from this router (directly from him or from my upstream > >> provider) can't reach the servers because the router still have only 1 > >> route going directly to his bge2 interface (the interface with carp) > >> and he have no clue of the MASTER interface. > >> > >> Maybe I'm worng and OSPF is not the solution. > >> > >> What I try to do is to have a redundant gateway for my servers (CARP) > >> and I want to have 2 upstreams provider with BGP (multihoming) > >> > >> I need a way for this 2 routers to talk to each other and share their > >> internal routes to know how to reach both of the "exit" point (route > >> to both upstream provider) and how to reach the MASTER interface of > >> every CARP group. > >> > >> Any idea? > >> > > > >If you are just running with two routers you don't need to use OSPF. > >Use CARP for the inside network, setup the upstream sessions on each > >router (perhaps even using "depend on carp" to fail over the sessions) and > >setup a IBGP session between the two routers -- best via a dedicated > >interface. Set "set nexthop self" on the IBGP sessions and you should be > >fine. > > > >-- > >:wq Claudio
Re: carp, ospf can't see carp state
Hi Claudio, I have double check on my lab and everything work fine for the OSPF part, sorry for my mistake. But at the end, I'm still having the same problem: the server didn't know the right route. OSPF see all the route correctly but the system didn't seem to be updated. If I do "route show" I only see the local route pointing directly to the CARP device instead of pointing to the other router. route show give me something like this when my cable is unplug from the "carp" interface: 83.201.77/24link#10UC 0 0 - carp1 What do you think it can be? Thanks, FranC'ois 2007/4/7, Claudio Jeker <[EMAIL PROTECTED]>: On Sat, Apr 07, 2007 at 12:21:19PM -0400, Frangois Rousseau wrote: > But how I'm suppose to annonce the route for the right carp interface? > Right now my servers can always reach the router because of the CARP > interface but the router can't always reach the servers... > > If I unplug the cable of my CARP interface (bge2 for example), all > traffic from this router (directly from him or from my upstream > provider) can't reach the servers because the router still have only 1 > route going directly to his bge2 interface (the interface with carp) > and he have no clue of the MASTER interface. > > Maybe I'm worng and OSPF is not the solution. > > What I try to do is to have a redundant gateway for my servers (CARP) > and I want to have 2 upstreams provider with BGP (multihoming) > > I need a way for this 2 routers to talk to each other and share their > internal routes to know how to reach both of the "exit" point (route > to both upstream provider) and how to reach the MASTER interface of > every CARP group. > > Any idea? > If you are just running with two routers you don't need to use OSPF. Use CARP for the inside network, setup the upstream sessions on each router (perhaps even using "depend on carp" to fail over the sessions) and setup a IBGP session between the two routers -- best via a dedicated interface. Set "set nexthop self" on the IBGP sessions and you should be fine. -- :wq Claudio
Re: carp, ospf can't see carp state
FranC'ois Rousseau wrote:
>> > But how I'm suppose to annonce the route for the right carp interface?
>> > Right now my servers can always reach the router because of the CARP
>> > interface but the router can't always reach the servers...
>> >
>> > If I unplug the cable of my CARP interface (bge2 for example), all
>> > traffic from this router (directly from him or from my upstream
>> > provider) can't reach the servers because the router still have only 1
>> > route going directly to his bge2 interface (the interface with carp)
>> > and he have no clue of the MASTER interface.
>> >
>> > Maybe I'm worng and OSPF is not the solution.
>> >
>> > What I try to do is to have a redundant gateway for my servers (CARP)
>> > and I want to have 2 upstreams provider with BGP (multihoming)
>> >
>> > I need a way for this 2 routers to talk to each other and share their
>> > internal routes to know how to reach both of the "exit" point (route
>> > to both upstream provider) and how to reach the MASTER interface of
>> > every CARP group.
>> >
>> > Any idea?
>> >
Your situation is different from mine, I am new to OSPF, and my
information may not help you any, but here it is:
I have a set up with two external routers and two internal routers. Both
external routers uplink to the same ISP unlike in your situation. They
share a carp'd external/inet IP and the status of this carp interface
(and other path/interface failures determines which external router is
used as the main uplink. My main problem setting this up is somewhat
similar to yours in terms of getting the internal routers to know which
external router to use for default route/external ISP access. The key
for me was to have the ospf directives "redistribute connected" and
"redistribute default" in the external routers' ospf.conf. Then I made
sure that the internal routers did NOT have a statically assigned
default route by removing /etc/mygate (since static routes take
precedence over ospf-learned routes). This enabled me to have failover
of my external/uplink routers.
External router ospf.conf:
primaryInlink="bge0"
backupInlink="bge1"
inet="carp0"
dmz="carp1"
# global configuration
router-id 0.0.0.40
fib-update yes
redistribute connected
redistribute default
auth-type crypt
auth-md 1 scrubbedForPosting
auth-md-keyid 1
# areas
area 0 {
interface $primaryInlink {
}
interface $backupInlink {
metric 100
}
interface $inet {
passive
}
interface $dmz {
passive
}
}
The dual Inlinks are because my setup is fully connected via dedicated
links, all inter-router traffic only goes through these dedicated pair
links, not through a switch.
Hope this helps,
Chris
Re: carp, ospf can't see carp state
Hi Claudio, In fact, I'm looking to had a third providers soon (maybe 4-5 weeks) so I will start to use 3 routers. I also want to use OSPF for future expension. I have one interface by router dedicated to inter-router traffic (iBGP, OSPF and of course the data to the other upstream provider) I have read your documents "Routing with OpenBSD using OpenOSPFD and OpenBGPD" and "OpenOSPFD" (http://www.networx.ch/OpenOSPFD%20-%20Presentation.pdf) and I'm under the impression that is possible to announce the MASTER interface of a CARP group with OpenOSPFD. "OpenOSPFD will honour the state of the carp(4) interface and only the router that is carp master will announce the network to the other routers. There is one problem left OpenOSPFD is not able to preempt the carp(4) interface if one of the OSPF links goes down. Currently the best way to solve this issue is to add a direct connection between the two carp routers." "Instead use carp to connect a LAN with servers to an OSPF cloud more than one ospf router default gateway on servers is carped and does not change Use a "passive" carp interface and multiple ethernet interfaces to connect router to the OSPF cloud; link-state of carp interface is tracked route in the OSPF cloud will always point to the active carp interface" I want to be able to loose my connection to Upstream provider 1 and to loose my connection to the "lan" on router2 without loosing the connection to Internet. In case my explanation are not clear, I have put a draft of my network here: http://step.polymtl.ca/~spock/draft.jpg Do you think is possible to do this with OpenOSPFD ? I have also reread your first post, if I understand correctly, it's already supposed to work? I will double check monday when I will be in front of my lab but friday when I have check, the routers are not advertise the carp route to the other router. Thanks for your help, FranC'ois 2007/4/7, Claudio Jeker <[EMAIL PROTECTED]>: On Sat, Apr 07, 2007 at 12:21:19PM -0400, Frangois Rousseau wrote: > But how I'm suppose to annonce the route for the right carp interface? > Right now my servers can always reach the router because of the CARP > interface but the router can't always reach the servers... > > If I unplug the cable of my CARP interface (bge2 for example), all > traffic from this router (directly from him or from my upstream > provider) can't reach the servers because the router still have only 1 > route going directly to his bge2 interface (the interface with carp) > and he have no clue of the MASTER interface. > > Maybe I'm worng and OSPF is not the solution. > > What I try to do is to have a redundant gateway for my servers (CARP) > and I want to have 2 upstreams provider with BGP (multihoming) > > I need a way for this 2 routers to talk to each other and share their > internal routes to know how to reach both of the "exit" point (route > to both upstream provider) and how to reach the MASTER interface of > every CARP group. > > Any idea? > If you are just running with two routers you don't need to use OSPF. Use CARP for the inside network, setup the upstream sessions on each router (perhaps even using "depend on carp" to fail over the sessions) and setup a IBGP session between the two routers -- best via a dedicated interface. Set "set nexthop self" on the IBGP sessions and you should be fine. -- :wq Claudio
Re: carp, ospf can't see carp state
On Sat, Apr 07, 2007 at 12:21:19PM -0400, Frangois Rousseau wrote: > But how I'm suppose to annonce the route for the right carp interface? > Right now my servers can always reach the router because of the CARP > interface but the router can't always reach the servers... > > If I unplug the cable of my CARP interface (bge2 for example), all > traffic from this router (directly from him or from my upstream > provider) can't reach the servers because the router still have only 1 > route going directly to his bge2 interface (the interface with carp) > and he have no clue of the MASTER interface. > > Maybe I'm worng and OSPF is not the solution. > > What I try to do is to have a redundant gateway for my servers (CARP) > and I want to have 2 upstreams provider with BGP (multihoming) > > I need a way for this 2 routers to talk to each other and share their > internal routes to know how to reach both of the "exit" point (route > to both upstream provider) and how to reach the MASTER interface of > every CARP group. > > Any idea? > If you are just running with two routers you don't need to use OSPF. Use CARP for the inside network, setup the upstream sessions on each router (perhaps even using "depend on carp" to fail over the sessions) and setup a IBGP session between the two routers -- best via a dedicated interface. Set "set nexthop self" on the IBGP sessions and you should be fine. -- :wq Claudio
Re: carp, ospf can't see carp state
But how I'm suppose to annonce the route for the right carp interface?
Right now my servers can always reach the router because of the CARP
interface but the router can't always reach the servers...
If I unplug the cable of my CARP interface (bge2 for example), all
traffic from this router (directly from him or from my upstream
provider) can't reach the servers because the router still have only 1
route going directly to his bge2 interface (the interface with carp)
and he have no clue of the MASTER interface.
Maybe I'm worng and OSPF is not the solution.
What I try to do is to have a redundant gateway for my servers (CARP)
and I want to have 2 upstreams provider with BGP (multihoming)
I need a way for this 2 routers to talk to each other and share their
internal routes to know how to reach both of the "exit" point (route
to both upstream provider) and how to reach the MASTER interface of
every CARP group.
Any idea?
Thanks,
Henning Brauer: I will try to do it at the end of my lab, but I'm new
in openBSD and I'm short in time for this project.
2007/4/7, Claudio Jeker <[EMAIL PROTECTED]>:
On Fri, Apr 06, 2007 at 06:38:01PM -0400, FranC'ois Rousseau wrote:
> Hi,
>
> I'm configuring 2 server to use as a gateway for multihoming.
>
> I use:
>
> OpenBSD 4.0 stable and OpenBSD 4.0 release
> OpenBGPD
> OpenOSPFD
> CARP for failover
>
> I have 2 router with 3 interfaces and 5 carp interfaces by router.
>
> interface1 = eBGP with 2 upstream provider
> interface2 = Link between the 2 routers, OSPF, iBGP (10.10.10.0/30)
> (em0 and bge0)
> interface3 = 5 * carp --> use as gateway for my servers (bge2)
>
> Right now the BGP, and the CARP work correctly.
>
> My problem is with the OSPF part. OSPF by itself work correctly but
> it didn't announce any route concerning the carp interface.
>
> If I start ospfd with "ospfd -dv" I see many:
> "if_fsm: event UP resulted in action START and changing state for
> interface carpX from DOWN to DOWN"
>
That's normal. Carp interfaces are always DOWN aka passive because it is
impossible to run OSPF over a carp interface.
The routes covered by carp are included in the router LSA as stub
networks. ospfctl show data router and ospfctl show rib will show these
networks.
--
:wq Claudio
> ---
>
> Router1:
> ospfd.conf
>
> router-id 0.0.0.1
> redistribute 44.25.32.41/30
>
> area 0.0.0.0 {
> auth-type crypt
> auth-md 1 "iii"
> auth-md 2 "jjj"
> auth-md-keyid 1
>
> interface em0 {
> metric 10
> }
> interface carp0 {
> passive
> }
> interface carp1
> interface carp2
> interface carp3
> interface carp4
> }
>
>
>
> hostname.bge2:
> inet 83.201.76.2 255.255.255.0 NONE description "My network"
>
> -
>
> Router2:
> ospfd.conf
>
> router-id 0.0.0.2
> redistribute 211.6.17.17/30
>
> area 0.0.0.0 {
> auth-type crypt
> auth-md 1 "iii"
> auth-md 2 "jjj"
> auth-md-keyid 1
>
> interface bge0
> interface carp0 {
> passive
> }
> interface carp1
> interface carp2
> interface carp3
> interface carp4
> }
>
>
>
> hostname.bge2:
> inet 23.182.158.2 255.255.255.0 NONE description "My network"
>
>
> ---
>
> Both router:
>
> CARP:
> hostname.carp0:
> inet 23.182.158.1 255.255.255.0 23.182.158.255 vhid 1 pass
> temppass2 carpdev bge2
>
> hostname.carp1:
>inet 83.201.77.1 255.255.255.0 83.201.77.255 vhid2 pass temppass2
> carpdev bge2
>
> hostname.carp2:
>inet 83.201.78.1 255.255.255.0 83.201.78.255 vhid3 pass temppass2
> carpdev bge2
>
> hostname.carp3:
>inet 83.201.79.1 255.255.255.0 83.201.79.255 vhid4 pass temppass2
> carpdev bge2
>
> hostname.carp4:
> inet 83.201.76.1 255.255.255.0 83.201.76.255 vhid5 pass temppass2 carpdev
> bge2
>
>
>
> Thanks,
> FranC'ois
Re: carp, ospf can't see carp state
On Fri, Apr 06, 2007 at 06:38:01PM -0400, Frangois Rousseau wrote:
> Hi,
>
> I'm configuring 2 server to use as a gateway for multihoming.
>
> I use:
>
> OpenBSD 4.0 stable and OpenBSD 4.0 release
> OpenBGPD
> OpenOSPFD
> CARP for failover
>
> I have 2 router with 3 interfaces and 5 carp interfaces by router.
>
> interface1 = eBGP with 2 upstream provider
> interface2 = Link between the 2 routers, OSPF, iBGP (10.10.10.0/30)
> (em0 and bge0)
> interface3 = 5 * carp --> use as gateway for my servers (bge2)
>
> Right now the BGP, and the CARP work correctly.
>
> My problem is with the OSPF part. OSPF by itself work correctly but
> it didn't announce any route concerning the carp interface.
>
> If I start ospfd with "ospfd -dv" I see many:
> "if_fsm: event UP resulted in action START and changing state for
> interface carpX from DOWN to DOWN"
>
That's normal. Carp interfaces are always DOWN aka passive because it is
impossible to run OSPF over a carp interface.
The routes covered by carp are included in the router LSA as stub
networks. ospfctl show data router and ospfctl show rib will show these
networks.
--
:wq Claudio
> ---
>
> Router1:
> ospfd.conf
>
> router-id 0.0.0.1
> redistribute 44.25.32.41/30
>
> area 0.0.0.0 {
> auth-type crypt
> auth-md 1 "iii"
> auth-md 2 "jjj"
> auth-md-keyid 1
>
> interface em0 {
> metric 10
> }
> interface carp0 {
> passive
> }
> interface carp1
> interface carp2
> interface carp3
> interface carp4
> }
>
>
>
> hostname.bge2:
> inet 83.201.76.2 255.255.255.0 NONE description "My network"
>
> -
>
> Router2:
> ospfd.conf
>
> router-id 0.0.0.2
> redistribute 211.6.17.17/30
>
> area 0.0.0.0 {
> auth-type crypt
> auth-md 1 "iii"
> auth-md 2 "jjj"
> auth-md-keyid 1
>
> interface bge0
> interface carp0 {
> passive
> }
> interface carp1
> interface carp2
> interface carp3
> interface carp4
> }
>
>
>
> hostname.bge2:
> inet 23.182.158.2 255.255.255.0 NONE description "My network"
>
>
> ---
>
> Both router:
>
> CARP:
> hostname.carp0:
> inet 23.182.158.1 255.255.255.0 23.182.158.255 vhid 1 pass
> temppass2 carpdev bge2
>
> hostname.carp1:
>inet 83.201.77.1 255.255.255.0 83.201.77.255 vhid2 pass temppass2
> carpdev bge2
>
> hostname.carp2:
>inet 83.201.78.1 255.255.255.0 83.201.78.255 vhid3 pass temppass2
> carpdev bge2
>
> hostname.carp3:
>inet 83.201.79.1 255.255.255.0 83.201.79.255 vhid4 pass temppass2
> carpdev bge2
>
> hostname.carp4:
> inet 83.201.76.1 255.255.255.0 83.201.76.255 vhid5 pass temppass2 carpdev
> bge2
>
>
>
> Thanks,
> FranC'ois
Re: carp, ospf can't see carp state
this might be related to carp's incorrect way oif dealing with routes.
can peole please test this diff for me and report back? it allows bgpd
to act correctly even wen a carp interface was not master at the time
of bgpd startup. it will fix other problems too, it might fix yours. if
people don't test this it'll just rot in my tree...
tech talk: it adds the missing route messages upon route
insertion/deletion.
Index: ip_carp.c
===
RCS file: /cvs/src/sys/netinet/ip_carp.c,v
retrieving revision 1.135
diff -u -p -r1.135 ip_carp.c
--- ip_carp.c 27 Mar 2007 21:58:16 - 1.135
+++ ip_carp.c 28 Mar 2007 23:18:51 -
@@ -368,15 +368,18 @@ carp_setroute(struct carp_softc *sc, int
struct ifaddr *ifa;
int s;
+ /* XXX this mess needs fixing */
+
s = splsoftnet();
TAILQ_FOREACH(ifa, &sc->sc_if.if_addrlist, ifa_list) {
switch (ifa->ifa_addr->sa_family) {
case AF_INET: {
- int count = 0;
+ int count = 0, error;
struct sockaddr sa;
struct rtentry *rt;
struct radix_node_head *rnh;
struct radix_node *rn;
+ struct rt_addrinfo info;
int hr_otherif, nr_ourif;
/*
@@ -395,9 +398,15 @@ carp_setroute(struct carp_softc *sc, int
}
/* Remove the existing host route, if any */
- rtrequest(RTM_DELETE, ifa->ifa_addr,
- ifa->ifa_addr, ifa->ifa_netmask,
- RTF_HOST, NULL, 0);
+ bzero(&info, sizeof(info));
+ info.rti_info[RTAX_DST] = ifa->ifa_addr;
+ info.rti_info[RTAX_GATEWAY] = ifa->ifa_addr;
+ info.rti_info[RTAX_NETMASK] = ifa->ifa_netmask;
+ info.rti_flags = RTF_HOST;
+ error = rtrequest1(RTM_DELETE, &info, NULL, 0);
+ rt_missmsg(RTM_DELETE, &info, info.rti_flags, NULL,
+ error, 0);
+
/* Check for our address on another interface */
/* XXX cries for proper API */
@@ -420,26 +429,39 @@ carp_setroute(struct carp_softc *sc, int
if (hr_otherif) {
ifa->ifa_rtrequest = NULL;
ifa->ifa_flags &= ~RTF_CLONING;
-
- rtrequest(RTM_ADD, ifa->ifa_addr,
- ifa->ifa_addr, ifa->ifa_netmask,
- RTF_UP | RTF_HOST, NULL, 0);
+ bzero(&info, sizeof(info));
+ info.rti_info[RTAX_DST] = ifa->ifa_addr;
+ info.rti_info[RTAX_GATEWAY] =
ifa->ifa_addr;
+ info.rti_info[RTAX_NETMASK] =
ifa->ifa_netmask;
+ info.rti_flags = RTF_UP | RTF_HOST;
+ error = rtrequest1(RTM_ADD, &info,
NULL, 0);
+ rt_missmsg(RTM_ADD, &info,
info.rti_flags, NULL,
+ error, 0);
}
if (!hr_otherif || nr_ourif || !rt) {
if (nr_ourif && !(rt->rt_flags &
- RTF_CLONING))
- rtrequest(RTM_DELETE, &sa,
- ifa->ifa_addr,
- ifa->ifa_netmask, 0, NULL,
- 0);
+ RTF_CLONING)) {
+ bzero(&info, sizeof(info));
+ info.rti_info[RTAX_DST] = &sa;
+ info.rti_info[RTAX_GATEWAY] =
ifa->ifa_addr;
+ info.rti_info[RTAX_NETMASK] =
ifa->ifa_netmask;
+ error = rtrequest1(RTM_DELETE,
&info, NULL, 0);
+ rt_missmsg(RTM_DELETE, &info,
info.rti_flags, NULL,
+ error, 0);
+ }
ifa->ifa_rtrequest = arp_rtrequest;
ifa->ifa_flags |= RTF_CLONING;
- if (rtrequest(RTM_ADD, ifa->ifa_addr,
- ifa->ifa_addr,
carp, ospf can't see carp state
Hi,
I'm configuring 2 server to use as a gateway for multihoming.
I use:
OpenBSD 4.0 stable and OpenBSD 4.0 release
OpenBGPD
OpenOSPFD
CARP for failover
I have 2 router with 3 interfaces and 5 carp interfaces by router.
interface1 = eBGP with 2 upstream provider
interface2 = Link between the 2 routers, OSPF, iBGP (10.10.10.0/30)
(em0 and bge0)
interface3 = 5 * carp --> use as gateway for my servers (bge2)
Right now the BGP, and the CARP work correctly.
My problem is with the OSPF part. OSPF by itself work correctly but
it didn't announce any route concerning the carp interface.
If I start ospfd with "ospfd -dv" I see many:
"if_fsm: event UP resulted in action START and changing state for
interface carpX from DOWN to DOWN"
---
Router1:
ospfd.conf
router-id 0.0.0.1
redistribute 44.25.32.41/30
area 0.0.0.0 {
auth-type crypt
auth-md 1 "iii"
auth-md 2 "jjj"
auth-md-keyid 1
interface em0 {
metric 10
}
interface carp0 {
passive
}
interface carp1
interface carp2
interface carp3
interface carp4
}
hostname.bge2:
inet 83.201.76.2 255.255.255.0 NONE description "My network"
-
Router2:
ospfd.conf
router-id 0.0.0.2
redistribute 211.6.17.17/30
area 0.0.0.0 {
auth-type crypt
auth-md 1 "iii"
auth-md 2 "jjj"
auth-md-keyid 1
interface bge0
interface carp0 {
passive
}
interface carp1
interface carp2
interface carp3
interface carp4
}
hostname.bge2:
inet 23.182.158.2 255.255.255.0 NONE description "My network"
---
Both router:
CARP:
hostname.carp0:
inet 23.182.158.1 255.255.255.0 23.182.158.255 vhid 1 pass
temppass2 carpdev bge2
hostname.carp1:
inet 83.201.77.1 255.255.255.0 83.201.77.255 vhid2 pass temppass2
carpdev bge2
hostname.carp2:
inet 83.201.78.1 255.255.255.0 83.201.78.255 vhid3 pass temppass2
carpdev bge2
hostname.carp3:
inet 83.201.79.1 255.255.255.0 83.201.79.255 vhid4 pass temppass2
carpdev bge2
hostname.carp4:
inet 83.201.76.1 255.255.255.0 83.201.76.255 vhid5 pass temppass2 carpdev bge2
Thanks,
FranC'ois
