Re: multilink VPN
On 2009-05-31, Stuart Henderson wrote: >> Uhm. The tunnel endpoints and the gre src and dest IPs are the same. I >> have a bad feeling about that. > > ahh, changing that gets me a lot further, thanks. gre's nasty hack > to toggle the address's LSB isn't quite enough then; not a problem. > >> Additionally I remember some strange issues with gre(4) and ospfd but I >> thought I fixed that. > > let's see if I can work out how to get beyond EXCHG... ah, that just needed the gre0 config clearing from ospfd (reload with the line commented and reload with it back in) and now it's fine and appears to be stable. good. thanks for the clue Claudio.
Re: multilink VPN
On 2009-05-31, Claudio Jeker wrote:
> On Sun, May 31, 2009 at 01:13:25PM +, Stuart Henderson wrote:
>> On 2009-05-31, Stuart Henderson wrote:
>> > On 2009-05-29, Stuart Henderson wrote:
>> >>
>> >> OSPF over gre's or gif's (which can then themselves be protected by
>> >> ipsec) is probably the fastest option at present on OpenBSD.
>> >
>> > Hrmm. And then I try it...
>> >
>> > Does anyone actually have this working and if so would they mind
>> > sharing config? I'm seeing the hellos go out the physical interface
>> > rather than the gre.
>> >
>> > # tcpdump -nivr0 -vv proto ospf
>> > 13:00:18.661860 85.158.44.158 > 224.0.0.5: OSPFv2-hello 44: rtrid
>> > 85.158.44.149 backbone auth MD5 E mask 255.255.255.255 int 1 pri 1 dead 4
>> > nbrs [tos 0xc0] [ttl 1] (id 53330, len 80)
>> > 13:00:19.672022 85.158.44.158 > 224.0.0.5: OSPFv2-hello 44: rtrid
>> > 85.158.44.149 backbone auth MD5 E mask 255.255.255.255 int 1 pri 1 dead 4
>> > nbrs [tos 0xc0] [ttl 1] (id 23013, len 80)
>> > 13:00:20.682184 85.158.44.158 > 224.0.0.5: OSPFv2-hello 44: rtrid
>> > 85.158.44.149 backbone auth MD5 E mask 255.255.255.255 int 1 pri 1 dead 4
>> > nbrs [tos 0xc0] [ttl 1] (id 23179, len 80)
>> > 13:00:21.692350 85.158.44.158 > 224.0.0.5: OSPFv2-hello 44: rtrid
>> > 85.158.44.149 backbone auth MD5 E mask 255.255.255.255 int 1 pri 1 dead 4
>> > nbrs [tos 0xc0] [ttl 1] (id 60275, len 80)
>> >
>> > # tcpdump -nigre0 -vv proto ospf
>> >
>> >
>> > The gre itself is fine, I can ping over it and the packets show up
>> > correctly on gre0, and also correctly on vr0 marked with "(gre encap)".
>> > It's correct (per RFC2328 8.1) that AllSPFRouters is used rather than
>> > the tunnel endpoint addresses even on point-to-point.
>> >
>> >
>>
>> Hrmm and double hrmm.
>>
>> startup
>> orig_rtr_lsa: area 0.0.0.0
>> orig_rtr_lsa: stub net, interface vr0
>> if_fsm: event UP resulted in action START and changing state for interface
>> vr0 from DOWN to WAIT
>> if_join_group: error IP_ADD_MEMBERSHIP, interface gre0 address 224.0.0.5:
>> Address already in use
>>
>> $ sudo grep -A3 area /etc/ospfd.conf
>>
>> area 0.0.0.0 {
>> interface gre0 { metric 200 }
>> interface vr0
>> }
>>
>> $ ifconfig gre0
>> gre0: flags=9011 mtu 1476
>> priority: 0
>> groups: gre
>> physical address inet 85.158.44.158 --> 195.95.187.1
>> inet6 fe80::20d:b9ff:fe13:5198%gre0 -> prefixlen 64 scopeid 0x6
>> inet 85.158.44.158 --> 195.95.187.1 netmask 0x
>>
>
> Uhm. The tunnel endpoints and the gre src and dest IPs are the same. I
> have a bad feeling about that.
ahh, changing that gets me a lot further, thanks. gre's nasty hack
to toggle the address's LSB isn't quite enough then; not a problem.
> Additionally I remember some strange issues with gre(4) and ospfd but I
> thought I fixed that.
let's see if I can work out how to get beyond EXCHG...
Re: multilink VPN
On Sun, May 31, 2009 at 01:13:25PM +, Stuart Henderson wrote:
> On 2009-05-31, Stuart Henderson wrote:
> > On 2009-05-29, Stuart Henderson wrote:
> >>
> >> OSPF over gre's or gif's (which can then themselves be protected by
> >> ipsec) is probably the fastest option at present on OpenBSD.
> >
> > Hrmm. And then I try it...
> >
> > Does anyone actually have this working and if so would they mind
> > sharing config? I'm seeing the hellos go out the physical interface
> > rather than the gre.
> >
> > # tcpdump -nivr0 -vv proto ospf
> > 13:00:18.661860 85.158.44.158 > 224.0.0.5: OSPFv2-hello 44: rtrid
> > 85.158.44.149 backbone auth MD5 E mask 255.255.255.255 int 1 pri 1 dead 4
> > nbrs [tos 0xc0] [ttl 1] (id 53330, len 80)
> > 13:00:19.672022 85.158.44.158 > 224.0.0.5: OSPFv2-hello 44: rtrid
> > 85.158.44.149 backbone auth MD5 E mask 255.255.255.255 int 1 pri 1 dead 4
> > nbrs [tos 0xc0] [ttl 1] (id 23013, len 80)
> > 13:00:20.682184 85.158.44.158 > 224.0.0.5: OSPFv2-hello 44: rtrid
> > 85.158.44.149 backbone auth MD5 E mask 255.255.255.255 int 1 pri 1 dead 4
> > nbrs [tos 0xc0] [ttl 1] (id 23179, len 80)
> > 13:00:21.692350 85.158.44.158 > 224.0.0.5: OSPFv2-hello 44: rtrid
> > 85.158.44.149 backbone auth MD5 E mask 255.255.255.255 int 1 pri 1 dead 4
> > nbrs [tos 0xc0] [ttl 1] (id 60275, len 80)
> >
> > # tcpdump -nigre0 -vv proto ospf
> >
> >
> > The gre itself is fine, I can ping over it and the packets show up
> > correctly on gre0, and also correctly on vr0 marked with "(gre encap)".
> > It's correct (per RFC2328 8.1) that AllSPFRouters is used rather than
> > the tunnel endpoint addresses even on point-to-point.
> >
> >
>
> Hrmm and double hrmm.
>
> startup
> orig_rtr_lsa: area 0.0.0.0
> orig_rtr_lsa: stub net, interface vr0
> if_fsm: event UP resulted in action START and changing state for interface
> vr0 from DOWN to WAIT
> if_join_group: error IP_ADD_MEMBERSHIP, interface gre0 address 224.0.0.5:
> Address already in use
>
> $ sudo grep -A3 area /etc/ospfd.conf
>
> area 0.0.0.0 {
> interface gre0 { metric 200 }
> interface vr0
> }
>
> $ ifconfig gre0
> gre0: flags=9011 mtu 1476
> priority: 0
> groups: gre
> physical address inet 85.158.44.158 --> 195.95.187.1
> inet6 fe80::20d:b9ff:fe13:5198%gre0 -> prefixlen 64 scopeid 0x6
> inet 85.158.44.158 --> 195.95.187.1 netmask 0x
>
Uhm. The tunnel endpoints and the gre src and dest IPs are the same. I
have a bad feeling about that. Additionally I remember some strange issues
with gre(4) and ospfd but I thought I fixed that.
--
:wq Claudio
Re: multilink VPN
On 2009-05-31, Stuart Henderson wrote:
> On 2009-05-29, Stuart Henderson wrote:
>>
>> OSPF over gre's or gif's (which can then themselves be protected by
>> ipsec) is probably the fastest option at present on OpenBSD.
>
> Hrmm. And then I try it...
>
> Does anyone actually have this working and if so would they mind
> sharing config? I'm seeing the hellos go out the physical interface
> rather than the gre.
>
> # tcpdump -nivr0 -vv proto ospf
> 13:00:18.661860 85.158.44.158 > 224.0.0.5: OSPFv2-hello 44: rtrid
> 85.158.44.149 backbone auth MD5 E mask 255.255.255.255 int 1 pri 1 dead 4
> nbrs [tos 0xc0] [ttl 1] (id 53330, len 80)
> 13:00:19.672022 85.158.44.158 > 224.0.0.5: OSPFv2-hello 44: rtrid
> 85.158.44.149 backbone auth MD5 E mask 255.255.255.255 int 1 pri 1 dead 4
> nbrs [tos 0xc0] [ttl 1] (id 23013, len 80)
> 13:00:20.682184 85.158.44.158 > 224.0.0.5: OSPFv2-hello 44: rtrid
> 85.158.44.149 backbone auth MD5 E mask 255.255.255.255 int 1 pri 1 dead 4
> nbrs [tos 0xc0] [ttl 1] (id 23179, len 80)
> 13:00:21.692350 85.158.44.158 > 224.0.0.5: OSPFv2-hello 44: rtrid
> 85.158.44.149 backbone auth MD5 E mask 255.255.255.255 int 1 pri 1 dead 4
> nbrs [tos 0xc0] [ttl 1] (id 60275, len 80)
>
> # tcpdump -nigre0 -vv proto ospf
>
>
> The gre itself is fine, I can ping over it and the packets show up
> correctly on gre0, and also correctly on vr0 marked with "(gre encap)".
> It's correct (per RFC2328 8.1) that AllSPFRouters is used rather than
> the tunnel endpoint addresses even on point-to-point.
>
>
Hrmm and double hrmm.
startup
orig_rtr_lsa: area 0.0.0.0
orig_rtr_lsa: stub net, interface vr0
if_fsm: event UP resulted in action START and changing state for interface vr0
from DOWN to WAIT
if_join_group: error IP_ADD_MEMBERSHIP, interface gre0 address 224.0.0.5:
Address already in use
$ sudo grep -A3 area /etc/ospfd.conf
area 0.0.0.0 {
interface gre0 { metric 200 }
interface vr0
}
$ ifconfig gre0
gre0: flags=9011 mtu 1476
priority: 0
groups: gre
physical address inet 85.158.44.158 --> 195.95.187.1
inet6 fe80::20d:b9ff:fe13:5198%gre0 -> prefixlen 64 scopeid 0x6
inet 85.158.44.158 --> 195.95.187.1 netmask 0x
$ route -n get 195.95.187.1
route to: 195.95.187.1
destination: 195.95.187.1
interface: gre0
if address: 85.158.44.158
priority: 4 (connected)
flags:
use mtuexpire
118 0 0
$ route -n get 195.95.187.0
route to: 195.95.187.0
destination: default
mask: default
gateway: 85.158.44.145
interface: vr0
if address: 85.158.44.158
priority: 8 (static)
flags:
use mtuexpire
151 0 0
Re: multilink VPN
On 2009-05-29, Stuart Henderson wrote: > > OSPF over gre's or gif's (which can then themselves be protected by > ipsec) is probably the fastest option at present on OpenBSD. Hrmm. And then I try it... Does anyone actually have this working and if so would they mind sharing config? I'm seeing the hellos go out the physical interface rather than the gre. # tcpdump -nivr0 -vv proto ospf 13:00:18.661860 85.158.44.158 > 224.0.0.5: OSPFv2-hello 44: rtrid 85.158.44.149 backbone auth MD5 E mask 255.255.255.255 int 1 pri 1 dead 4 nbrs [tos 0xc0] [ttl 1] (id 53330, len 80) 13:00:19.672022 85.158.44.158 > 224.0.0.5: OSPFv2-hello 44: rtrid 85.158.44.149 backbone auth MD5 E mask 255.255.255.255 int 1 pri 1 dead 4 nbrs [tos 0xc0] [ttl 1] (id 23013, len 80) 13:00:20.682184 85.158.44.158 > 224.0.0.5: OSPFv2-hello 44: rtrid 85.158.44.149 backbone auth MD5 E mask 255.255.255.255 int 1 pri 1 dead 4 nbrs [tos 0xc0] [ttl 1] (id 23179, len 80) 13:00:21.692350 85.158.44.158 > 224.0.0.5: OSPFv2-hello 44: rtrid 85.158.44.149 backbone auth MD5 E mask 255.255.255.255 int 1 pri 1 dead 4 nbrs [tos 0xc0] [ttl 1] (id 60275, len 80) # tcpdump -nigre0 -vv proto ospf The gre itself is fine, I can ping over it and the packets show up correctly on gre0, and also correctly on vr0 marked with "(gre encap)". It's correct (per RFC2328 8.1) that AllSPFRouters is used rather than the tunnel endpoint addresses even on point-to-point.
Re: multilink VPN
James Mackinnon wrote on Friday, May 29, 2009 6:25 PM > Hi All > > Thanks for your feedback. > > The guy regarding the cisco is a CCIE so I tend to accept his > statements > quick enough.. > > In VPN, I am referencing it in general terms in the creation of a > private > network over a public network of course. I would go with MPLS or > another > technology, however again, not 100% failsafe. > > Their application is a thick app which has allowances for network > drops, > however, the data is a real-time life and death type of solution in > that > they are a security monitoring company with multiple sites to which > access > data in 1 location. This is what I must ensure stays up because staff > must > be able to handle the alarms.. > > Roughly 1 million alarms a day go through this network, thus, any > outage can > result in dropped alarms.. Our solutions in both facilities also offer > some > allowances for drops by caching an alarm until network return, however > applications failures are also bad in this case. > > At first, I was looking at BGP, and in the past have used it, but with > convergence time on a net down situation, it doesn't come close to the > time > required. > > Personally, I think any solution that can rebuild in 10-30 seconds is a > very > solid solution. If they are not happy with that, I could recommend a > very > expensive alternative but that won't fly. > > Stuart, do you know of some sources I should review on your mentioned > idea. > > I am also looking at multi-segmenting the locations systems and having > their > applications account for loss to failover to the second IP. > > fun little project, very small to almost nil budget is the challange. > > Cheers If it absolutely has to be up, OpenVMS
Re: multilink VPN
In cisco speak, with pretty pictures: http://www.cisco.com/en/US/tech/tk583/tk372/technologies_configuration_example09186a00800a43f6.shtml On OpenBSD, it works analoguously, except that it's much cleaner :) Just think of the ipsec secured gre tunnel as a wire from point A to B. Make two such wires. Then run a routing protocol on them to redundantly route your traffic through. Some things to consider: 1. What if the internet links fail some other way than completely dead, like high packet loss? 2. The rest of the system probably isn't as reliable as you think, if you can't have much money for making the internet links redundant. -- Jussi Peltola
Re: multilink VPN
Hi All Thanks for your feedback. The guy regarding the cisco is a CCIE so I tend to accept his statements quick enough.. In VPN, I am referencing it in general terms in the creation of a private network over a public network of course. I would go with MPLS or another technology, however again, not 100% failsafe. Their application is a thick app which has allowances for network drops, however, the data is a real-time life and death type of solution in that they are a security monitoring company with multiple sites to which access data in 1 location. This is what I must ensure stays up because staff must be able to handle the alarms.. Roughly 1 million alarms a day go through this network, thus, any outage can result in dropped alarms.. Our solutions in both facilities also offer some allowances for drops by caching an alarm until network return, however applications failures are also bad in this case. At first, I was looking at BGP, and in the past have used it, but with convergence time on a net down situation, it doesn't come close to the time required. Personally, I think any solution that can rebuild in 10-30 seconds is a very solid solution. If they are not happy with that, I could recommend a very expensive alternative but that won't fly. Stuart, do you know of some sources I should review on your mentioned idea. I am also looking at multi-segmenting the locations systems and having their applications account for loss to failover to the second IP. fun little project, very small to almost nil budget is the challange. Cheers James - Original Message - From: "Stuart Henderson" To: Sent: Friday, May 29, 2009 7:37 PM Subject: Re: multilink VPN On 2009-05-29, Toni Mueller wrote: On Wed, 27.05.2009 at 22:07:25 -0300, James Mackinnon wrote: I need to setup redundant VPN's between these locations without the use of BGP. I have used sasync in the past, pfsync etc however, I have not tried to setup a VPN where 2 ISPs are used without the ISPs setup with BGP. Because BGP convergance can take a bit of time, and the network in this case not being able to drop for 1 second, I need to determine what option is best. I heavily doubt that you'll be able to keep the network up at all times because even CARP failover will take longer than one second. OSPF over gre's or gif's (which can then themselves be protected by ipsec) is probably the fastest option at present on OpenBSD. You're restricted to the lowest value you can set router-dead-time to; with very aggressive timers (which are likely to cause problems with false drops) that's 2 seconds. 3-4 seconds (with hellos at a second) is more realistic for fast recovery over ethernet or some good quality pseudowire circuit. Not sure exactly what you mean by "VPN" as it's not a well defined term but you should look at that carefully. e.g. Rekeying can be a little on the slow side, you want to avoid this happening on both connections at the same time. I strongly suspect that if you really want to force less than 1 seconds of downtime even in the case of error, then you need to swap IP for a real high-reliability type of connection like telcos use in their long hauls (eg. SDH). BFD can be quite quick. In some parts of the world these better types of connection are simply not available. If you're used to what's available in Europe (1Gb ethernet-presented private circuit over about 15 miles for GBP21K/year?) you will find the situation in some places absolutely unbelievable.
Re: multilink VPN
On 2009-05-29, Toni Mueller wrote: > On Wed, 27.05.2009 at 22:07:25 -0300, James Mackinnon > wrote: >> I need to setup redundant VPN's between these locations without the use of >> BGP. > >> I have used sasync in the past, pfsync etc however, I have not tried to setup >> a VPN where 2 ISPs are used without the ISPs setup with BGP. Because BGP >> convergance can take a bit of time, and the network in this case not being >> able to drop for 1 second, I need to determine what option is best. > > I heavily doubt that you'll be able to keep the network up at all > times because even CARP failover will take longer than one second. OSPF over gre's or gif's (which can then themselves be protected by ipsec) is probably the fastest option at present on OpenBSD. You're restricted to the lowest value you can set router-dead-time to; with very aggressive timers (which are likely to cause problems with false drops) that's 2 seconds. 3-4 seconds (with hellos at a second) is more realistic for fast recovery over ethernet or some good quality pseudowire circuit. Not sure exactly what you mean by "VPN" as it's not a well defined term but you should look at that carefully. e.g. Rekeying can be a little on the slow side, you want to avoid this happening on both connections at the same time. > I strongly suspect that if you really want to force less than 1 seconds > of downtime even in the case of error, then you need to swap IP for a > real high-reliability type of connection like telcos use in their long > hauls (eg. SDH). BFD can be quite quick. In some parts of the world these better types of connection are simply not available. If you're used to what's available in Europe (1Gb ethernet-presented private circuit over about 15 miles for GBP21K/year?) you will find the situation in some places absolutely unbelievable.
Re: multilink VPN
Hi,
On Wed, 27.05.2009 at 22:07:25 -0300, James Mackinnon
wrote:
> I need to setup redundant VPN's between these locations without the use of
> BGP.
> I have used sasync in the past, pfsync etc however, I have not tried to setup
> a VPN where 2 ISPs are used without the ISPs setup with BGP. Because BGP
> convergance can take a bit of time, and the network in this case not being
> able to drop for 1 second, I need to determine what option is best.
I heavily doubt that you'll be able to keep the network up at all
times because even CARP failover will take longer than one second.
> I have spoke with a cisco guy today and they can do multilink VPN's on cisco
> for this,
Did he actually tell you how they make sure that there'll be no
downtime of even one second? Was the explanation technically sound?
How about error conditions in the Internet, between your sites?
FWIW, I've configured semi-"multilink" VPN in the past (before the
"CARP age"), with this kind of setup:
LAN1 --- FW{1,2} --- Internet --- FW{3,4} --- LAN2
with
LAN1, FW1, FW2: my end
FW3, FW4, LAN2: other end (not accessible to me)
Manually switching between FW1 and FW2 usually took on the order of
8-15 seconds.
The other side switched between FW3 and FW4 at their leisure, w/o
telling anyone.
The idea to configure this with isakmpd.conf was to have both peers
configured on both of your firewalls, and then add as many IPSEC
connections so that you cover all connection pairs.
That way, you can access LAN2 from LAN1 regardless whether FW3 or FW4
is operational. In my setup, one of the tunnels simply vanished and the
other appeared, if the other side switched their firewalls.
Now, if you can detect your conditions under which you want to fail
over to the other firewall (eg. fiber cut), it should be easy to
cook up a script and fire it on such an event.
But you won't get away without any downtime, and if you find out how to
do this on the IP level, I'm interested to hear about it.
I strongly suspect that if you really want to force less than 1 seconds
of downtime even in the case of error, then you need to swap IP for a
real high-reliability type of connection like telcos use in their long
hauls (eg. SDH).
But if you can weed out duplicate packets, you might be able to create
some magic with bridging and move all packets over both links all the
time, dropping one half at the receiving end(s). But this is only a
shot in the dark - I don't know how to do this.
I'm curious about what kind of application you have that does not
tolerate 1 second of downtime?
If someone has an idea about how to configure this with ipsec.conf, I'm
eager to hear.
Kind regards,
--Toni++
multilink VPN
Hi All Here is my situation and I am hoping for a little guidance on this one I have 2 locations, both with 2 fiber internet connections I need to setup redundant VPN's between these locations without the use of BGP. So, my setup would be something like this Location A Firewall 1 Connection to ISP1 Wan IP 24.22.22.1 Firewall 2 Connection to ISP2 Wan IP 33.33.33.1 Internal Interfaces are in a carp setup Internal IP range is 192.168.0.0/24 Location B Firewall 1 Connection to ISP1 Wan IP 24.22.21.1 Firewall 2 Connection to ISP2 Wan IP 33.33.32.1 Internal Interfaces are in a carp setup Internal IP Range is 192.168.1.0/24 I have used sasync in the past, pfsync etc however, I have not tried to setup a VPN where 2 ISPs are used without the ISPs setup with BGP. Because BGP convergance can take a bit of time, and the network in this case not being able to drop for 1 second, I need to determine what option is best. I have spoke with a cisco guy today and they can do multilink VPN's on cisco for this, however, being a bit of a OpenBSD fan and prefer to use Openbsd over cisco any time I can, I would really like to accomplish this task using OpenBSD. Thoughts or direction would be great james
