Re: Performance of Wireguard on Infiniband 40G
Hey again, On Sat, May 13, 2017 at 9:37 AM, Baptiste Jonglezwrote: > - Debian jessie > - Linux 3.16.43-2 One small and unfortunate thought just occurred to me: the backporting to really old kernels I'm pretty sure is way less efficient than newer kernels on the RX, due to some missing core fast-path APIs in the old kernels. In particular, I had to wrap the UDP layer with some nasty hacks to get packets out, whereas newer kernels have an elegant API for that which integrates in the right place. Just a thought... I haven't actually done concrete measurements though. Jason ___ WireGuard mailing list WireGuard@lists.zx2c4.com https://lists.zx2c4.com/mailman/listinfo/wireguard
Re: TX counters not updated on wireguard interface
Okay I figured it out. Upstream 039f50629b7f860f36644ed1f34b27da9aa62f43 only came in 4.5, so explicit calls to iptunnel_xmit_stats is required. I'll play with this and see if I can fix it. Jason ___ WireGuard mailing list WireGuard@lists.zx2c4.com https://lists.zx2c4.com/mailman/listinfo/wireguard
Re: TX counters not updated on wireguard interface
Hey Baptiste, I'll try to reproduce this shortly. I did make a change to some compat code [1] for kernels older than 3.12 to fix a similar bug. But Jesse has a newer kernel than that, I think? I'll check into it and try to fix. Thanks for reporting. Jason [1] https://git.zx2c4.com/WireGuard/patch/?id=1370d96432c2c7e86705ce99b1682a3b7c2ce1cd ___ WireGuard mailing list WireGuard@lists.zx2c4.com https://lists.zx2c4.com/mailman/listinfo/wireguard
Re: Performance of Wireguard on Infiniband 40G
Hey Baptiste, Awesome test! Thanks for reporting the results. On Sat, May 13, 2017 at 9:37 AM, Baptiste Jonglezwrote: > Using iperf (TCP mode) over the wireguard interface, performance was > around 1.6 Gbit/s. In bidirectional mode (iperf -d), performance was > 700 Mbit/s + 800 Mbit/s. Indeed the current multicore algorithm has a lot of issues. Samuel, CCd, is going to be doing some work on optimizing this algorithm this summer. > After raising the MTU of the wireguard interface to 65450, performance > went up to 7.6 Gbit/s (unidirectional iperf). It makes sense that it'd be higher, since CPUs work best when running uninterrupted, but still this indicates that padata is a very suboptimal algorithm. Expect some improvements on this in the coming months. Hopefully you'll be able to test on similar hardware at some point when things are finished. > Note that infiniband has a MTU of 65520 bytes, but Wireguard still selects > a MTU of 1420 bytes for its interface. Yea the 1420 is just a hard coded "default". I probably add something clever to autoselect an MTU when configuring the first peer's first endpoint (by computing the route and taking its interface's mtu and doing subtraction, etc), but the long term solution, I think, will be to do some more clever PMTU situation from within WireGuard. I'm still working out exactly how to do this, but it should be possible. > - Xeon E5520 @2.27GHz (2 CPUs, 4 cores each) > - Mellanox ConnectX IB 4X QDR MT26428 *drools* That's some awesome hardware! > - iperf 2.0.5 iperf2 has the -b bidirectional mode which is nice, but it seems like most people are using iperf3 now. Out of curiosity, is there a reason for preferring iperf2, beyond the -b switch? Jason ___ WireGuard mailing list WireGuard@lists.zx2c4.com https://lists.zx2c4.com/mailman/listinfo/wireguard
Performance of Wireguard on Infiniband 40G
Hi, Just for information, I did a quick test of Wireguard over a 40G Infiniband network, between two machines with a Xeon E5520. Using iperf (TCP mode) over the wireguard interface, performance was around 1.6 Gbit/s. In bidirectional mode (iperf -d), performance was 700 Mbit/s + 800 Mbit/s. Note that infiniband has a MTU of 65520 bytes, but Wireguard still selects a MTU of 1420 bytes for its interface. After raising the MTU of the wireguard interface to 65450, performance went up to 7.6 Gbit/s (unidirectional iperf). Using the Infiniband network directly, iperf's performance is 21.7 Gbit/s (iperf maxes out the CPU at the receiver, even when using 8 threads). Hardware used: - Xeon E5520 @2.27GHz (2 CPUs, 4 cores each) - Mellanox ConnectX IB 4X QDR MT26428 Versions used: - Debian jessie - Linux 3.16.43-2 - Wireguard 0.0.20170421-2 - iperf 2.0.5 - Mellanox ConnectX InfiniBand driver v2.2-1 Baptiste signature.asc Description: PGP signature ___ WireGuard mailing list WireGuard@lists.zx2c4.com https://lists.zx2c4.com/mailman/listinfo/wireguard
Rebasing Wireguard's master branch
Hi Jason, Could you please stop rebasing the master branch of the git repository? It's really annoying (and possibly confusing) to obtain merge conflicts while simply pulling the latest changes. If you want to rebase, please use feature branches or something similar. Thanks, Baptiste signature.asc Description: PGP signature ___ WireGuard mailing list WireGuard@lists.zx2c4.com https://lists.zx2c4.com/mailman/listinfo/wireguard
TX counters not updated on wireguard interface
Hi, While trying 0.0.20170421-2 on Debian jessie, I noticed that the TX counters are not updated: $ ip -s link show 7: wg:mtu 1420 qdisc noqueue state UNKNOWN mode DEFAULT group default link/none RX: bytes packets errors dropped overrun mcast 4384059324 4073303 0 0 0 0 TX: bytes packets errors dropped carrier collsns 0 00 0 0 0 This is after a bidirectional TCP iperf, so they has been some trafic in both directions. Bandwidth monitors like bmon also don't show any outgoing trafic. Baptiste signature.asc Description: PGP signature ___ WireGuard mailing list WireGuard@lists.zx2c4.com https://lists.zx2c4.com/mailman/listinfo/wireguard
Installation instructions for Debian
Hi Jason, Could you update the installation instructions for Debian [1]? Based on the discussion from a few months ago, and given that Wireguard now supports Jessie's 3.16 kernel, these instructions should be added: For Debian jessie or stretch: # echo "deb http://deb.debian.org/debian/ unstable main" > /etc/apt/sources.list.d/unstable-wireguard.list # cat < /etc/apt/preferences.d/limit-unstable Package: * Pin: release a=unstable Pin-Priority: 200 EOF # apt update # apt install wireguard-dkms wireguard-tools I have verified that the resulting setup is functional (with 0.0.20170421-2 on jessie). Thanks, Baptiste [1] https://www.wireguard.io/install/ signature.asc Description: PGP signature ___ WireGuard mailing list WireGuard@lists.zx2c4.com https://lists.zx2c4.com/mailman/listinfo/wireguard