[Openvpn-devel] Fwd: RE: Multi-channel VPN

[James Yonan]

I'm forwarding this discussion of an interesting feature request.  Namely,
could (and should) OpenVPN have a channel bonding capability, where more than
one UDP connection over different paths is used to connect two peers, and
OpenVPN does channel bonding, load balancing, and failover among the
connections?  Or does this function not belong in OpenVPN since it is
functionally distinct from the role of network security and tunneling and
could be implemented as a module or driver apart from OpenVPN?

James

Forwarded From: "R. Latimer" 

> Hi,
> 
> Certainly, I don't mind if you wish to post to the list.
> 
> My reason for suggesting this be done in OpenVPN is primarily due to the
> ability to support multiple platforms (my e-mail should say
> platform-independant, not hardware-independant) without relying on
> additional software being installed at both ends, or OS-specific solutions.
> 
> Thanks,
> 
> - R. Latimer
> 
> -Original Message-
> From: James Yonan [mailto:j...@yonan.net]
> Sent: Thursday, 17 April 2003 22:40
> To: R. Latimer
> Subject: Re: Multi-channel VPN
> 
> 
> Mr. Latimer,
> 
> It's an interesting idea, though I'm not sure that OpenVPN is the right
> place
> to put the channel bonding code.
> 
> Since OpenVPN leaves all routing control to the OS, it would seem to make
> more
> sense to implement the channel bonding at the router level, since channel
> bonding is functionally distinct from the role of a VPN.
> 
> Having said that, it would certainly be possible to implement this in
> OpenVPN
> though it would add the complexity of load balancing, failover, to OpenVPN.
> It would also introduce issues of flow control on the UDP channel(s) that
> OpenVPN hasn't had to deal with before.
> 
> If you don't mind, I would like to post this thread to the list so that
> others
> can comment.
> 
> James
> 
> "R. Latimer"  said:
> 
> > Hi James,
> >
> > Sorry to e-mail you directly rather than using the mailing list. I'm not
> > presently subscribed to the development list, but have a feature
> suggestion
> > you may wish to give some consideration (Post-1.4, perhaps much further
> down
> > the development path).
> >
> > I was wondering if it would be possible to allow OpenVPN to use multiple
> > paths to connect two networks? E.g. use two Internet connections here to
> > connect to a single remote VPN. I see no reason both ends couldn't use
> > multiple paths, but a simple one to many relationship alone would be
> handy.
> >
> > The reason for this is, Internet access in NZ is very expensive. I have a
> > 128k connection, and am looking at getting 128k a wireless connection as
> > well for use with a laptop. The VPN terminates in the US, where bandwidth
> is
> > plentiful, and supplying the VPN with 256k wouldn't be a problem.
> >
> > I would like to be able to join the two channels when the laptop is
> > connected to the LAN, and when unavailable, have all traffic automatically
> > sent through a single interface.
> >
> > Each interface would be with a different ISP, so traditional routing
> > protocols to one IP address wouldn't work in this case. In fact, the local
> > telephone company specifically disallows static IPs on the 128k
> > connections - they are only permitted on the expensive (pay per MB)
> > high-speed ADSL connections.
> >
> > I believe this would be possible using two instances of OpenVPN, and
> > NetGraph on FreeBSD, but unfortunately the remote end uses Linux. If
> > possible to do this in OpenVPN, it would provide a hardware-independant
> way
> > to link multiple channels. Currently NetGraph contains no code for
> > intelligently determining the path, it simply alternates between ports. It
> > would be great if OpenVPN could additionally favour one link over another
> > based on actual performance.
> >
> > I've been using OpenVPN since the FreeBSD port became available, and it
> > works very well. Thanks for such a great tool, and I hope you'll give my
> > suggestion some thought.
> >
> > Thanks,
> >
> > - R. Latimer
> >
> >
> 
> 
> 
> --
> 
> 
> 
> 
> 
> 



-- 

Re: [Openvpn-devel] Re: New beta available + progress update

[James Yonan]

Matthias Andree  said:

> On Thu, 17 Apr 2003, James Yonan wrote:
> 
> > A better alternative (orginally suggested by you) is to avoid fragmenting in
> > the first place by bouncing back ICMP_DEST_UNREACH/ICMP_FRAG_NEEDED to the 
> > TUN
> > device.  This won't work on TAP devices because the ethernet MTU is fixed
at 1500.
> > 
> > Unfortunately (or fortunately depending on your perspective), my network
> > connections are not broken enough to benefit from the FRAGMENT_ENABLE stuff
> > just yet, so I can't really test it properly.
> 
> Well, the peer that was behind an overloaded 1 Mbps 2 km radio link has
> now moved and has become a V.34+ modem link (33k6 bps), with
> corresponding effects on the MTU. I wonder if running a local pppd pair
> with properly MTU can be used to figure if fragmentation works, but
> there's yet have to be someone who drops individual fragments (unless
> gremlin mode does it already) for real testing.

Gremlin tests that the dynamic MTU/bandwidth algorithm can survive, but
doesn't really help when it comes to tuning the algorithm for optimality in
real-world conditions.

The nice part about a radio link is that it is probably under your control,
meaning that you can ensure that ICMPs get properly passed.  This allows path
MTU discovery to work and therefore solves a lot of the harder problems.

In this case, the FRAGMENT_ENABLE code would take the Path MTU hint from the
OS rather than trying to figure it out empirically by trial-and-error.  The
current FRAGMENT_ENABLE code knows how to get the PMTU from Linux, but still
needs code for other OSes.

> OTOH, fragmenting links also have a natural impact on the packet loss:
> if you assume 10% packet loss on the transport layer (that transports
> the fragment), and you fragment a TAP packet of 1500+ bytes into 3 of
> approx 500 bytes (say, MTU=552), the effective packet loss for TCP or
> something is much higher: it takes losing a single fragment to
> retransmit, so the effective packet loss will come out as 27% (not
> taking burst loss into account).

Good point.

> I also have another patch to fix prototype warnings:

Merged.

James

> 
> diff -u thread.h thread.h
> --- thread.h  17 Apr 2003 10:51:21 -
> +++ thread.h  17 Apr 2003 11:05:38 -
> @@ -118,17 +118,17 @@
>  #define MUTEX_UNLOCK(lock)
>  
>  static inline void
> -thread_init()
> +thread_init(void)
>  {
>  }
>  
>  static inline void
> -thread_cleanup()
> +thread_cleanup(void)
>  {
>  }
>  
>  static inline int
> -thread_number()
> +thread_number(void)
>  {
>return 0;
>  }
> @@ -139,7 +139,7 @@
>  }
>  
>  static inline void
> -work_thread_join ()
> +work_thread_join (void)
>  {
>  }
>  
> 
> -- 
> Matthias Andree
> 
> 
> ---
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> ___
> Openvpn-devel mailing list
> Openvpn-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/openvpn-devel
> 



-- 

Re: [Openvpn-devel] Re: New beta available + progress update

[James Yonan]

Matthias Andree  said:

> > http://openvpn.sourceforge.net/beta/openvpn-1.3.2.21.tar.gz (or CVS)
> 
> I have a next round of patches to fix prototypes and types to quench
> compiler warnings and get a more robust source code against changed
> environments, to aid possible later debugging; it also includes GCC
> extensions (only used when GCC is being used) that allow stricter msg()
> and buf_printf() format string vs. argument checking, to avoid printing
> bogus data when type sizes differ (think 64 bit).

Yes, very good ideas.  I will merge.

> 
> One problem remains after applying the patch:
> openvpn.c: In function `openvpn':
> openvpn.c:470: warning: cast discards qualifiers from pointer target type

That's a tricky matter, because the struct options object is const, but it
needs to be passed to a thread, and normally you need to cast something to a
void* if you want to pass it to a thread as an argument, which then causes the
constness to be lost.

> Other than that, we have unused parameter warnings (you said before
> you're not fixing them), shadow parameter warnings (variable names being
> the same as system functions, such as time or nice)

I'm not too worried about the shadow parameters.  I think it's pretty obvious
when something is being used as a variable vs. a function call.

As for the unused parms, they are mostly a result of conditional compilation
making certain variables unnecessary.  OpenVPN has a large conditional
compilation state-space, so I'm not sure that it makes sense to try to
eliminate all such warnings.

> and in particular,
> aggregate (struct) values in function calls and function returns, these
> are inefficient (as the compiler must toss the whole struct on the
> stack, which involves copy overhead), and some of them (buffer.c in
> particular) look pretty suspicious. Is there any C standard that
> guarantees that returning an auto struct works and copies the return
> value?

While I can't quote chapter and verse, being able to manipulate structures in
a value context, such as returning them from functions or passing them on the
stack, has been a part of C for ages, and for a long time compilers have tried
very hard to optimize this usage as much as possible.

For example, when a function returns an automatic structure, most compilers
will optimize this into a zero-copy operation, where the functions's
references to the automatic structure go directly into a variable allocated by
the caller for its return.

My attitude towards optimization has been to optimize the code across the
critical path only, which is essentially the event loop in openvpn.c.  Code
outside the event loop (such as the SSL negotiation) does not need to be
optimized with respect to CPU usage, because it is executed infrequently in
relation to the main event loop and OpenSSL EVP crypto code.

Most of the expensive buffer.[ch] code is only executed on startup, shutdown,
SIGHUP, or when the debug level is high.

[ patch deleted ]

James

[Openvpn-devel] Re: New beta available + progress update

[Matthias Andree]

On Thu, 17 Apr 2003, James Yonan wrote:

> A better alternative (orginally suggested by you) is to avoid fragmenting in
> the first place by bouncing back ICMP_DEST_UNREACH/ICMP_FRAG_NEEDED to the TUN
> device.  This won't work on TAP devices because the ethernet MTU is fixed at 
> 1500.
> 
> Unfortunately (or fortunately depending on your perspective), my network
> connections are not broken enough to benefit from the FRAGMENT_ENABLE stuff
> just yet, so I can't really test it properly.

Well, the peer that was behind an overloaded 1 Mbps 2 km radio link has
now moved and has become a V.34+ modem link (33k6 bps), with
corresponding effects on the MTU. I wonder if running a local pppd pair
with properly MTU can be used to figure if fragmentation works, but
there's yet have to be someone who drops individual fragments (unless
gremlin mode does it already) for real testing.

OTOH, fragmenting links also have a natural impact on the packet loss:
if you assume 10% packet loss on the transport layer (that transports
the fragment), and you fragment a TAP packet of 1500+ bytes into 3 of
approx 500 bytes (say, MTU=552), the effective packet loss for TCP or
something is much higher: it takes losing a single fragment to
retransmit, so the effective packet loss will come out as 27% (not
taking burst loss into account).

I also have another patch to fix prototype warnings:

diff -u thread.h thread.h
--- thread.h17 Apr 2003 10:51:21 -
+++ thread.h17 Apr 2003 11:05:38 -
@@ -118,17 +118,17 @@
 #define MUTEX_UNLOCK(lock)

 static inline void
-thread_init()
+thread_init(void)
 {
 }

 static inline void
-thread_cleanup()
+thread_cleanup(void)
 {
 }

 static inline int
-thread_number()
+thread_number(void)
 {
   return 0;
 }
@@ -139,7 +139,7 @@
 }

 static inline void
-work_thread_join ()
+work_thread_join (void)
 {
 }


-- 
Matthias Andree

[Openvpn-devel] Re: New beta available + progress update

[James Yonan]

Matthias Andree  said:

> > What the FRAGMENT_ENABLE code does is to add an extra 4 byte header to each
> > datagram that includes, among other things, feedback on the number of
> > datagrams received as well as the maximum datagram size received.  This
> > information can then be used by an OpenVPN peer to dynamically set the MTU
> > size as well as the datagram transmit rate independently of the OS and the
> > proper functioning of path MTU discovery.  Ultimately this code can make
> > OpenVPN more robust in situations where fragmentation is necessary (such as 
> > in
> > TAP-based bridged ethernets) when firewalls or routers in the path break 
> > PMTU
> > discovery.
> 
> How about compatibility? Does this need a FRAGMENT_ENABLEd OpenVPN on
> either side of the tunnel or will OpenVPN negotiate the fragment
> capability dynamically as well?

Negotiation of the fragment capability is similar to how other parameters are
negotiated, such as cipher algorithm, lzo compression, etc.  The option
(--mtu-dynamic) must be used on both ends of the connection.  In SSL mode,
both peers will complain if --mtu-dynamic is used on one end but not the
other.  In non-SSL mode, there is no negotiation, so each peer assumes that
the other is using compatible options.

> Does this FRAGMENT_ENABLE rely on SSL or
> will it work with preshared keys?

No, it doesn't rely on SSL or pre-shared keys.  It can work even on plaintext
tunnels.

I think the major issue still to be worked out is how to make it efficient. 
While I've written a simple heuristic (see fragment_adjust_mtu_bandwidth() in
fragment.c) that is a starting point for determining MTU and transmit throttle
levels based on feedback from the peer, I'm finding a big performance dropoff
once the fragmenting code kicks in.  A big reason for this is packet loss,
that can be seen if you run ftp over the tunnel.  When there is a one-to-one
correspondence between TUN/TAP packets and UDP datagrams (as there is in
non-FRAGMENT_ENABLE openvpn), then the TCP level does the heavy lifting in
terms of pushing packets at the optimal rate to avoid packet loss.  Once
openvpn starts doing its own fragmenting, then it also needs to take explicit
control over the transmit rate (as it can now do statically with --shaper).

A better alternative (orginally suggested by you) is to avoid fragmenting in
the first place by bouncing back ICMP_DEST_UNREACH/ICMP_FRAG_NEEDED to the TUN
device.  This won't work on TAP devices because the ethernet MTU is fixed at 
1500.

Unfortunately (or fortunately depending on your perspective), my network
connections are not broken enough to benefit from the FRAGMENT_ENABLE stuff
just yet, so I can't really test it properly.

James

[Openvpn-devel] Re: New beta available + progress update

[Matthias Andree]

> http://openvpn.sourceforge.net/beta/openvpn-1.3.2.21.tar.gz (or CVS)

I have a next round of patches to fix prototypes and types to quench
compiler warnings and get a more robust source code against changed
environments, to aid possible later debugging; it also includes GCC
extensions (only used when GCC is being used) that allow stricter msg()
and buf_printf() format string vs. argument checking, to avoid printing
bogus data when type sizes differ (think 64 bit).

One problem remains after applying the patch:
openvpn.c: In function `openvpn':
openvpn.c:470: warning: cast discards qualifiers from pointer target type

Other than that, we have unused parameter warnings (you said before
you're not fixing them), shadow parameter warnings (variable names being
the same as system functions, such as time or nice) and in particular,
aggregate (struct) values in function calls and function returns, these
are inefficient (as the compiler must toss the whole struct on the
stack, which involves copy overhead), and some of them (buffer.c in
particular) look pretty suspicious. Is there any C standard that
guarantees that returning an auto struct works and copies the return
value?

Index: buffer.c
===
RCS file: /cvsroot/openvpn/openvpn/buffer.c,v
retrieving revision 1.12
diff -u -r1.12 buffer.c
--- buffer.c16 Mar 2003 08:34:12 -  1.12
+++ buffer.c17 Apr 2003 10:51:16 -
@@ -111,7 +111,7 @@
  * printf append to a buffer with overflow check
  */
 void
-buf_printf (struct buffer *buf, char *format, ...)
+buf_printf (struct buffer *buf, const char *format, ...)
 {
   va_list arglist;

@@ -223,7 +223,7 @@

 char *
 format_hex_ex (const uint8_t *data, int size, int maxoutput,
-  int space_break, char* separator)
+  int space_break, const char* separator)
 {
   struct buffer out = alloc_buf_gc (maxoutput ? maxoutput :
((size * 2) + (size / space_break) + 2));
Index: buffer.h
===
RCS file: /cvsroot/openvpn/openvpn/buffer.h,v
retrieving revision 1.11
diff -u -r1.11 buffer.h
--- buffer.h17 Apr 2003 07:12:04 -  1.11
+++ buffer.h17 Apr 2003 10:51:16 -
@@ -47,7 +47,7 @@
 struct buffer alloc_buf (size_t size);
 struct buffer clone_buf (const struct buffer* buf);
 struct buffer alloc_buf_gc (size_t size);  /* allocate buffer with garbage 
collection */
-struct buffer clear_buf ();
+struct buffer clear_buf (void);
 void free_buf (struct buffer *buf);

 static inline bool
@@ -93,7 +93,7 @@

 /* Like strncpy but makes sure dest is always null terminated */
 static inline void
-strncpynt (char *dest, const char *src, int maxlen)
+strncpynt (char *dest, const char *src, size_t maxlen)
 {
   strncpy (dest, src, maxlen);
   if (maxlen > 0)
@@ -116,7 +116,11 @@
 /*
  * printf append to a buffer with overflow check
  */
-void buf_printf (struct buffer *buf, char *format, ...);
+void buf_printf (struct buffer *buf, const char *format, ...)
+#ifdef __GNUC__
+__attribute__ ((format (printf, 2, 3)))
+#endif
+;

 /*
  * remove trailing newline
@@ -149,7 +153,7 @@
  */
 char *
 format_hex_ex (const uint8_t *data, int size, int maxoutput,
-  int space_break, char* separator);
+  int space_break, const char* separator);

 static inline char *
 format_hex (const uint8_t *data, int size, int maxoutput)
@@ -353,7 +357,7 @@
 void x_gc_free (void *p);

 static inline int
-gc_new_level ()
+gc_new_level (void)
 {
   struct gc_thread* thread = _gc_thread[thread_number()];
   return ++thread->gc_level;
Index: common.h
===
RCS file: /cvsroot/openvpn/openvpn/common.h,v
retrieving revision 1.14
diff -u -r1.14 common.h
--- common.h17 Apr 2003 07:12:04 -  1.14
+++ common.h17 Apr 2003 10:51:16 -
@@ -41,8 +41,12 @@
  */
 #define counter_format"%10lu"
 #define ptr_format"0x%08zx"
-#define time_format   "%u"
+#define time_format   "%lu"
 #define fragment_type_format  "0x%08x"
+
+/* these are used to cast the arguments
+ * and MUST match the formats above */
+typedef unsigned long time_type;

 /*
  * Functions used for circular buffer index arithmetic.
Index: configure.ac
===
RCS file: /cvsroot/openvpn/openvpn/configure.ac,v
retrieving revision 1.64
diff -u -r1.64 configure.ac
--- configure.ac17 Apr 2003 07:12:04 -  1.64
+++ configure.ac17 Apr 2003 10:51:17 -
@@ -223,6 +223,9 @@
[],
[#include "syshead.h"])

+AC_CHECK_SIZEOF(unsigned int)
+AC_CHECK_SIZEOF(unsigned long)
+
 AC_CACHE_SAVE

 dnl check for other types
Index: crypto.h
===
RCS file: /cvsroot/openvpn/openvpn/crypto.h,v
retrieving revision 1.11
diff -u -r1.11 

[Openvpn-devel] New beta available + progress update

[James Yonan]

OpenVPN continues to evolve, and I thought I would take this opportunity to
briefly describe some of the current directions in the project (which,
incidentally, has passed its 1 year milestone).

For one, a new OpenVPN beta is available and testing would be appreciated.

http://openvpn.sourceforge.net/beta/openvpn-1.3.2.21.tar.gz (or CVS)

Thanks to the stability of 1.3.2, I've held off on making a new release that's
composed entirely of minor changes, and have instead focussed on some bigger
issues such as making the handling of MTU and fragmentation more dynamic and
automatic.  To this end, the new beta contains a number of improvements to
allow for dynamic MTU resizing.  This code is still experimental, and must be
explicitly enabled by defining FRAGMENT_ENABLE and rebuilding.

What the FRAGMENT_ENABLE code does is to add an extra 4 byte header to each
datagram that includes, among other things, feedback on the number of
datagrams received as well as the maximum datagram size received.  This
information can then be used by an OpenVPN peer to dynamically set the MTU
size as well as the datagram transmit rate independently of the OS and the
proper functioning of path MTU discovery.  Ultimately this code can make
OpenVPN more robust in situations where fragmentation is necessary (such as in
TAP-based bridged ethernets) when firewalls or routers in the path break PMTU
discovery.

My thinking right now is to make the next release 1.4.0 but leave the
FRAGMENT_ENABLE code off by default.  Even with the FRAGMENT_ENABLE code
disabled, there's enough new stuff here to justify a point release.

As well, there are a fair number of minor changes as well (see the ChangeLog
at the end of this message).

Looking post-1.4.0, the latest wishlist appears to be:

(1) Simplify configuration for setups that involve one server and numerous
roving clients (some of this has already been addressed by the --inetd option).

(2) Dynamic MTU support.  Ultimately, the FRAGMENT_ENABLE code will do this,
but it still needs testing and more developer input.

(3) Porting to Windows.  Judging by email I receive, there seems to be a lot
of demand for this.  In addition, a lot of developers have come forward who
are interested in working on a TUN/TAP driver for Windows (which is the
missing link), but so far I haven't seen much progress.  I may jump into the
fray and work on this myself, though due to the higher costs of developing for
Windows, I would need some corporate sponsorship to help defray costs.  Let me
know if you are interested in sponsoring such an undertaking.

If there are wishlist items I have missed, or if there is something you would
like to add, please let us know.

And also, please be reminded that the OpenVPN project is financially supported
by the user community.  Please consider donating to the project.  More info is
available here:

http://openvpn.sourceforge.net/donate.html

James

**

ChangeLog:

$Id: ChangeLog,v 1.69 2003/04/17 07:12:02 jimyonan Exp $

Upcoming version 1.4.0

* Added --replay-persist feature to allow replay
  protection across sessions.
* Fixed bug where --ifconfig could not be used
  with --tun-mtu.
* Added --tun-mtu-extra parameter to deal with
  the situation where a read on a TUN/TAP device
  returns more data than the device's MTU size.
* Fixed bug where some IPv6 support code for
  Linux was not being properly ifdefed out for
  Linux 2.2, causing compile errors.
* Added OPENVPN_EXIT_STATUS_x codes to
  openvpn.h to control which status value
  openvpn returns to its caller (such as
  a shell or inetd/xinetd) for various conditions.
* Added OPENVPN_DEBUG_COMMAND_LINE flag to
  openvpn.h to allow debugging in situations
  where stdout, stderr, and syslog cannot be used
  for message output, such as when OpenVPN is
  instantiated by inetd/xinetd.
* Removed owner-execute permission from file
  created by static key generator (Herbert Xu
  and Alberto Gonzalez Iniesta).
* Added --passtos option to allow IPv4 TOS bits
  to be passed from TUN/TAP input packets to
  the outgoing UDP socket (Craig Knox).
* Added code to prevent open socket file descriptors
  from being accessible to called scripts.
* Added --dev-name option (Christian Lademann).
* Added --mtu-disc option for manual control
  over MTU options.
* Show OS MTU value on UDP socket write failures
  (linux only).
* Numerous build system and portability
  fixes (Matthias Andree).
* Added better sensing of compiler support for
  variable argument macros, including (a) gcc
  style, (b) ISO C 1999 style, and (c) no support.
* Removed generated files from CVS.  Note INSTALL
  file for new CVS build commands.
* Changed all internal _* symbols to x_*
  for C standards compliance.
* Added TUN/TAP open code to cycle dynamically
  through unit numbers until it finds a free
  unit (based on code from Thomas Gielfeldt
  and VTun).
* Added dynamic MTU and fragmenting infrastructure
  (Experimental).  Rebuild with FRAGMENT_ENABLE
  defined