Re: [Openvpn-devel] TCP-over-TCP (was: Multi-channel VPN)

[James Yonan]

Aaron Sethman  said:

> 
> On Thu, 24 Apr 2003, James Yonan wrote:
> > Actually, I was thinking more about the situation where people are forced to
> > tunnel IP over TCP, for whatever reason, when UDP is not an option.  Since 
> > IP
> > is designed to transit over an unreliable physical layer, when you tunnel it
> > over TCP (a la vpnd or IP over ssh) you end up with this problem:
> >
> > http://sites.inka.de/sites/bigred/devel/tcp-tcp.html (see Stacking TCPs)
> I've read the paper many times, heh ;)
> 
> >
> > The key issue here is that when you tunnel IP over TCP, that will usually 
> > mean
> > TCP over TCP.  And this tends to break TCP's retransmission algorithm.  As 
> > the
> > article above states: "The upper layer will queue up more retransmissions
> > faster than the lower layer can process them".
> >
> > So my idea is why not filter out the upper layer TCP retransmissions, and
> > synthesize the return ACKs so that the upper TCP layer doesn't begin the
> > exponential backing off of the retransmit timer that eventually will stall 
> > the
> > connection?  This process would essentially strip the upper layer TCP of its
> > reliability function, since now the lower TCP layer will provide that
> > function.  This is not difficult to implement, since to OpenVPN, the upper
> > layer protocols are all simply byte streams flowing over the TUN pipe.  They
> > can be filtered without resorting to raw sockets or any other such 
> > potentially
> > unportable constructs.
> What if you end up filtering out necessary retransmissions that occur
> elsewhere on the path.  Consider the situation of one end being a router
> and passing packets out to the rest of the world, and openvpn is being
> used to brige an insecure link like say an 802.11b network.  Personally I
> think mucking with this could be dangerous.

That's a good point.  Once you synthesize an ACK to avoid retransmission over
the TCP-tunnel segment of the path, you are guaranteeing to the sender that
the packet has been received at the destination.  This means you would need to
do some accounting of TCP sequence numbers at both ends of the tunnel, so that
you could differentiate retransmits caused by local congestion on the
TCP-tunnel from retransmits occurring due to lossage elsewhere on the path. 
The more you look at it, the uglier it gets.

What you need is a kind of TCP mode that strips out the reliability function.
 Of course we have it, it's called UDP ;)

James

Re: [Openvpn-devel] TCP-over-TCP (was: Multi-channel VPN)

[Matthias Andree]

On Thu, 24 Apr 2003, James Yonan wrote:

> Actually, I was thinking more about the situation where people are forced to
> tunnel IP over TCP, for whatever reason, when UDP is not an option.  Since IP
> is designed to transit over an unreliable physical layer, when you tunnel it
> over TCP (a la vpnd or IP over ssh) you end up with this problem:
> 
> http://sites.inka.de/sites/bigred/devel/tcp-tcp.html (see Stacking TCPs)
> 
> The key issue here is that when you tunnel IP over TCP, that will usually mean
> TCP over TCP.  And this tends to break TCP's retransmission algorithm.  As the
> article above states: "The upper layer will queue up more retransmissions
> faster than the lower layer can process them".
> 
> So my idea is why not filter out the upper layer TCP retransmissions, and
> synthesize the return ACKs so that the upper TCP layer doesn't begin the
> exponential backing off of the retransmit timer that eventually will stall the
> connection?  This process would essentially strip the upper layer TCP of its

I think data volume (in flight or lost) is the issue, not stalls. Once
an ACK of the outer layer is returned, the inner layer will implicitly
also see an ACK (payload) which will then kick the data flow alive
again.

> reliability function, since now the lower TCP layer will provide that
> function.  This is not difficult to implement, since to OpenVPN, the upper
> layer protocols are all simply byte streams flowing over the TUN pipe.  They
> can be filtered without resorting to raw sockets or any other such potentially
> unportable constructs.

Only that OpenVPN would have to parse the TCP protocol for IPv4 and
IPv6.

-- 
Matthias Andree

Re: [Openvpn-devel] TCP-over-TCP (was: Multi-channel VPN)

[Aaron Sethman]

On Thu, 24 Apr 2003, James Yonan wrote:
> Actually, I was thinking more about the situation where people are forced to
> tunnel IP over TCP, for whatever reason, when UDP is not an option.  Since IP
> is designed to transit over an unreliable physical layer, when you tunnel it
> over TCP (a la vpnd or IP over ssh) you end up with this problem:
>
> http://sites.inka.de/sites/bigred/devel/tcp-tcp.html (see Stacking TCPs)
I've read the paper many times, heh ;)

>
> The key issue here is that when you tunnel IP over TCP, that will usually mean
> TCP over TCP.  And this tends to break TCP's retransmission algorithm.  As the
> article above states: "The upper layer will queue up more retransmissions
> faster than the lower layer can process them".
>
> So my idea is why not filter out the upper layer TCP retransmissions, and
> synthesize the return ACKs so that the upper TCP layer doesn't begin the
> exponential backing off of the retransmit timer that eventually will stall the
> connection?  This process would essentially strip the upper layer TCP of its
> reliability function, since now the lower TCP layer will provide that
> function.  This is not difficult to implement, since to OpenVPN, the upper
> layer protocols are all simply byte streams flowing over the TUN pipe.  They
> can be filtered without resorting to raw sockets or any other such potentially
> unportable constructs.
What if you end up filtering out necessary retransmissions that occur
elsewhere on the path.  Consider the situation of one end being a router
and passing packets out to the rest of the world, and openvpn is being
used to brige an insecure link like say an 802.11b network.  Personally I
think mucking with this could be dangerous.

> Using raw sockets would be another potential method -- instead of removing the
> reliability function from the upper layer TCP layer, remove it from the lower
> layer.  This would require a sort of fake-TCP implementation in user-space
> with the reliability function stripped out.  On first glance, this seems to be
> a more complicated approach because you would need to implement a lot of the
> TCP protocol in user space, and you would need to depend on the OS supporting
> raw sockets.  The approach outlined in the previous paragraph doesn't need raw
> sockets, and it doesn't need to reimplement TCP.  It only needs to examine the
> byte stream of IP packets flowing over the TUN device, filter out TCP
> retransmits, and synthesize return TCP ACKs.
This is what I was thinking, however this is a really big pain in the rear
too.

> Each method tries to stabilize TCP-over-TCP by eliminating a redundant
> reliability layer.  Both of these methods, while somewhat ugly, would
> eliminate a big part of the robustness issue of IP over TCP, for those who
> lack IP over UDP as an option.

Or you could just tell users that you are to expect degraded performance
in this case and use UDP if possible.  But *if* you are stuck, and have no
other options, this will work, but it'll suck too.

I'm wondering if there are some socket options one could set to twiddle
the TCP stack into playing slightly nicer.  I doubt it, because you've
still got the issue of other systems not playing nice with it.


-Aaron

Re: [Openvpn-devel] TCP-over-TCP (was: Multi-channel VPN)

[James Yonan]

Aaron Sethman  said:

> 
> 
> On Wed, 23 Apr 2003, Matthias Andree wrote:
> 
> > On Wed, 23 Apr 2003, James Yonan wrote:
> >
> > > I wonder if one could build a better tcp-over-tcp by doing some 
> > > intelligent
> > > packet filtering on the higher level tcp connection, such as filtering out
> > > retransmits and fudging return ACKs -- essentially removing those
elements of
> > > the TCP protocol which are designed to make TCP work over an unreliable
link.
> >
> > I wonder if that's necessary. Tunnelling through TCP is inherently
> > reliable no matter what you send, so TCP-nested-in-TCP is just overkill.
> > Cheating the OS doesn't help. Maybe some LD_PRELOAD library that turns
> > stream sockets into dgram sockets for connections that use the tunnel is
> > sufficient. However, this doesn't actually apply to openvpn because
> > openvpn does TCP-over-UDP.
> 
> I think you've got it backwards.  I think James is talking about the layer
> that openvpn is tunnelling over.  Basically fiddling with it to keep that
> layer from doing a lot of the reliabilitiy stuff.  To be honest I'm not
> sure its possible without using raw sockets and constructing your own TCP
> packets.  Even then though, I don't think that would work for some
> people's needs like being able to shove it through an SSL proxy or
> something like that.

Actually, I was thinking more about the situation where people are forced to
tunnel IP over TCP, for whatever reason, when UDP is not an option.  Since IP
is designed to transit over an unreliable physical layer, when you tunnel it
over TCP (a la vpnd or IP over ssh) you end up with this problem:

http://sites.inka.de/sites/bigred/devel/tcp-tcp.html (see Stacking TCPs)

The key issue here is that when you tunnel IP over TCP, that will usually mean
TCP over TCP.  And this tends to break TCP's retransmission algorithm.  As the
article above states: "The upper layer will queue up more retransmissions
faster than the lower layer can process them".

So my idea is why not filter out the upper layer TCP retransmissions, and
synthesize the return ACKs so that the upper TCP layer doesn't begin the
exponential backing off of the retransmit timer that eventually will stall the
connection?  This process would essentially strip the upper layer TCP of its
reliability function, since now the lower TCP layer will provide that
function.  This is not difficult to implement, since to OpenVPN, the upper
layer protocols are all simply byte streams flowing over the TUN pipe.  They
can be filtered without resorting to raw sockets or any other such potentially
unportable constructs.

Using raw sockets would be another potential method -- instead of removing the
reliability function from the upper layer TCP layer, remove it from the lower
layer.  This would require a sort of fake-TCP implementation in user-space
with the reliability function stripped out.  On first glance, this seems to be
a more complicated approach because you would need to implement a lot of the
TCP protocol in user space, and you would need to depend on the OS supporting
raw sockets.  The approach outlined in the previous paragraph doesn't need raw
sockets, and it doesn't need to reimplement TCP.  It only needs to examine the
byte stream of IP packets flowing over the TUN device, filter out TCP
retransmits, and synthesize return TCP ACKs.

Each method tries to stabilize TCP-over-TCP by eliminating a redundant
reliability layer.  Both of these methods, while somewhat ugly, would
eliminate a big part of the robustness issue of IP over TCP, for those who
lack IP over UDP as an option.

James