I've noticed some inconsistent performance with some of my tunnels and
thought I would take some of the spare free time I have over the holidays to
try to figure out what the cause of that may be. My environment in this
case is my home LAN.
Please forgive my use of the terms "server" and "client" in this email, I
only use these terms to simply explanation.
I statically assigned my server with an ip of 10.10.10.1, and my client is
set to 10.10.10.2. The rest of my LAN uses 192.168.2.0/24, so in this
case I am using Tinc to create a tunnel to access the 192.168.2.0/24 network
from my client. This is all on common switch fabric, no in-between
firewalls of any kind involved, and no firewalls configured on either Server
or Client.
On the Server, Tinc is running on stripped down Centos 5.5 as a virtual
machine and all numbers given here are in this configuration.
I have also tested this on a normal Centos 5.5 install, as well as Ubuntu
9.04, 9.10, 10.04, and 10.10. All with and without vmware tools installed.
Although there are performance differences observed between the different
builds, the behavior I describe has been the same on all builds. The only
thing I haven't tested is a native OS install.
Tinc is configured in switch mode.
The server virtual adapter is bridged to the physical adapter using brctl.
The client receives an address on the 192.168.2.0/24 network via DHCP from
my internet router.
ifconfig of Tinc "server"
br0 Link encap:Ethernet HWaddr 00:0C:29:58:B5:6B
inet addr:192.168.2.4 Bcast:192.168.255.255 Mask:255.255.0.0
inet6 addr: fe80::20c:29ff:fe58:b56b/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:21384 errors:0 dropped:0 overruns:0 frame:0
TX packets:23987 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:8452737 (8.0 MiB) TX bytes:23819155 (22.7 MiB)
br0:0 Link encap:Ethernet HWaddr 00:0C:29:58:B5:6B
inet addr:10.10.10.1 Bcast:10.10.10.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
eth0 Link encap:Ethernet HWaddr 00:0C:29:58:B5:6B
inet6 addr: fe80::20c:29ff:fe58:b56b/64 Scope:Link
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
RX packets:196191 errors:0 dropped:0 overruns:0 frame:0
TX packets:38068 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:145802768 (139.0 MiB) TX bytes:28914683 (27.5 MiB)
Interrupt:177 Base address:0x1424
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:222 errors:0 dropped:0 overruns:0 frame:0
TX packets:222 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:126456 (123.4 KiB) TX bytes:126456 (123.4 KiB)
vpn Link encap:Ethernet HWaddr FE:42:68:39:D9:1F
inet6 addr: fe80::fc42:68ff:fe39:d91f/64 Scope:Link
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
RX packets:13890 errors:0 dropped:0 overruns:0 frame:0
TX packets:22405 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:5055429 (4.8 MiB) TX bytes:21399229 (20.4 MiB)
[r...@localhost ~]# brctl show
bridge name bridge id STP enabled interfaces
br0 8000.000c2958b56b no vpn
eth0
ipconfig of windows xp "client"
Windows IP Configuration
Ethernet adapter Local Area Connection:
Connection-specific DNS Suffix . : local
IP Address. . . . . . . . . . . . . . . . : 10.10.10.2
Subnet Mask . . . . . . . . . . . . . . : 255.0.0.0
Default Gateway . . . . . . . . . . . . :
Ethernet adapter Wireless Network Connection 2:
Media State . . . . . . . . . . . : Media disconnected
Ethernet adapter Tinc:
Connection-specific DNS Suffix . : local
IP Address. . . . . . . . . . . . . : 192.168.2.246
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Default Gateway . . . . . . . . . : 192.168.2.1
What I've discovered using level 5 debugging is that often when a connection
is made, MTU probes from the client are not responded to.
The tell-tail sign I've seen every time is particularly high latency.
I've been able to reproduce the condition not every, but nearly every time,
if I manually start the client (windows xp client) in a command prompt.
Press Ctrl+c to stop the client, and then restart it after approximately 5
seconds.
The client will print the message "No response to MTU probes from Server"
And then basically all traffic from then point on carries the message
"Packet for Server (10.10.10.1 port 8002) larger than minimum MTU,
forwarding via TCP"
>From what I can tell, no further attempts at MTU probes are made, and the
connection will remain a TCP connection unless it is broken and restarted.
Usually if I stop the client and wait for about 30 seconds to reconnect,
there is a much greater chance that the MTU probes work fine, and in about
30 seconds MTU is fixed to 1416.
Every time when the MTU probing fails, I see latency between 700 - 1000 ms
with 32 byte pings over a LAN.
Every time when the MTU probing does not fail, I latency between 1 - 3 ms
with 32 byte pings over a LAN.
I used iperf to measure throughput in various configurations to compare.
The iperf server is a different device on the LAN.
don...@ubuntu:/opt/vmware/ovftool$ iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
** 3 tests directly on the lan, no VPN installed, as a baseline**
[ 4] local 192.168.2.31 port 5001 connected with 192.168.2.243 port 2826
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.0 sec 112 MBytes 93.4 Mbits/sec
[ 5] local 192.168.2.31 port 5001 connected with 192.168.2.243 port 2827
[ 5] 0.0-10.0 sec 110 MBytes 92.4 Mbits/sec
[ 4] local 192.168.2.31 port 5001 connected with 192.168.2.243 port 2828
[ 4] 0.0-10.0 sec 111 MBytes 93.1 Mbits/sec
** 4 tests over the VPN, MTU probing failed as observed using level 5
debugging**
[ 5] local 192.168.2.31 port 5001 connected with 192.168.2.246 port 2859
[ 5] 0.0-10.0 sec 232 KBytes 190 Kbits/sec
[ 4] local 192.168.2.31 port 5001 connected with 192.168.2.246 port 2862
[ 4] 0.0- 9.5 sec 1.33 MBytes 1.18 Mbits/sec
[ 5] local 192.168.2.31 port 5001 connected with 192.168.2.246 port 2863
[ 5] 0.0- 9.0 sec 1.48 MBytes 1.38 Mbits/sec
[ 4] local 192.168.2.31 port 5001 connected with 192.168.2.246 port 2864
[ 4] 0.0-10.5 sec 72.0 KBytes 56.0 Kbits/sec
** 5 tests over the VPN, MTU probling successful as obsserved using level 5
debugging, after MTU set to 1416 in the debug output **
[ 5] local 192.168.2.31 port 5001 connected with 192.168.2.246 port 2936
[ 5] 0.0-10.0 sec 19.3 MBytes 16.2 Mbits/sec
[ 4] local 192.168.2.31 port 5001 connected with 192.168.2.246 port 2938
[ 4] 0.0-10.0 sec 21.8 MBytes 18.3 Mbits/sec
[ 5] local 192.168.2.31 port 5001 connected with 192.168.2.246 port 2939
[ 5] 0.0-14.0 sec 8.41 MBytes 5.04 Mbits/sec
[ 4] local 192.168.2.31 port 5001 connected with 192.168.2.246 port 2942
[ 4] 0.0-10.0 sec 13.8 MBytes 11.6 Mbits/sec
[ 5] local 192.168.2.31 port 5001 connected with 192.168.2.246 port 2943
[ 5] 0.0-10.0 sec 14.2 MBytes 11.9 Mbits/sec
** 3 tests over the VPN without debugging to rule out performance loss due
to debugging. MTU believed to be successful because ping times were 1-3ms.
60 second delay before tests to allow MTU to settle **
[ 4] local 192.168.2.31 port 5001 connected with 192.168.2.246 port 2965
[ 4] 0.0-10.0 sec 21.9 MBytes 18.4 Mbits/sec
[ 5] local 192.168.2.31 port 5001 connected with 192.168.2.246 port 2966
[ 5] 0.0-10.0 sec 23.3 MBytes 19.5 Mbits/sec
[ 4] local 192.168.2.31 port 5001 connected with 192.168.2.246 port 2967
[ 4] 0.0-10.0 sec 22.2 MBytes 18.6 Mbits/sec
So as observed, when Tinc defaults to TCP due to MTU probes failing there is
a significant reduction in throughput and latency. When this happens in the
"real world" it's begun to become a little annoying to break and re-connect
several times until I get a good connection.
I haven't yet been able to figure out why the MTU probes only work some of
the time. I thought it might be my WAN but now I know it's not. There is
nothing in the path of this test to block them, and I haven't found any sign
of packet loss on the LAN.
I will do some testing later of setting PMTU directly and disabling
PMTUDiscovery to see if that will result in consistent behavior. I would
really rather leave dynamic PMTU though because it's a really nice feature
when you connect with a laptop and you never know what network medium you'll
be on.
_______________________________________________
tinc mailing list
tinc@tinc-vpn.org
http://www.tinc-vpn.org/cgi-bin/mailman/listinfo/tinc
