Re: Need for HW-clock independent timestamps

[Roman Mamedov]

On Thu, 17 May 2018 12:40:55 +0900
Paul  wrote:

> For me it looks like a problem solvable in software (as done for the 
> BMX routing protocol). Why even bother to get hardware involved?

Personally I am puzzled this is even an issue in WG. Not a single other VPN
protocol mandates every node to keep a monotonically increasing counter,
including even over reboots.

This has never been an issue in Tinc and OpenVPN at least, and if I'm not
mistaken neither in IPsec. And now suddenly we have people saying everyone now
has to buy and solder in some satellite based hardware just to use a VPN.

Given this didn't even arise in other VPN solutions, surely there must be other
way to solve the "replay attack" issue, without requiring an RTC (or a
persistent counter)? Perhaps nobody has just thought long enough about finding
one, and given the project in the early stages, just using the RTC (which
"everyone has") was chosen as a quick placeholder for now?

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: [Uber low priority] Linux 4.17.0 on aarch64

[Roman Mamedov]

On Thu, 07 Jun 2018 09:40:08 +0200
Riccardo Berto  wrote:

> Just want to report that I can't add a wg interface of type wireguard 
> with linux 4.17.0 on aarch64 (Raspberry Pi 3).
> 
> Error message: `RTNETLINK answers: Operation not supported`.
> 
> I'm using ArchLinuxARM. Downgrading to 4.16.x makes the issue dissapear. 
> This is not high-priority so please Jason, if you're reading this 
> message, move this issue further down in your debugging queue as I 
> believe almost nobody is using 4.17.0 on aarch64 right now. Maybe 
> they'll fix the error by themselves in the coming 4.17.x releases.

Did you actually compile the wireguard kernel module? Or rely on mysterious
"them" to do so?

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: Upstream Submission v1

[Roman Mamedov]

On Tue, 31 Jul 2018 21:26:53 +0200
"Jason A. Donenfeld"  wrote:

> Hey list,
> 
> I submitted patchset v1 of WireGuard to LKML a few minutes ago:
> 
> [0/3] https://marc.info/?l=linux-netdev=153306429108040=2
> [1/3] https://marc.info/?l=linux-netdev=153306429908043=2
> [2/3] https://marc.info/?l=linux-netdev=153306437408074=2
> [3/3] https://marc.info/?l=linux-netdev=153306440208084=2
> 
> I'm pretty elated, as getting v1 out the door marks a major milestone.
> 
> I expect for the first submission to be mercilessly criticized from
> all sorts of interesting and useful angles. There will most certainly
> be a v2, and it will be better because of whatever intense criticism
> received from v1. So hold on tight: a wild process is about to begin.

Congratulations, and a minor thing, please don't forget to CC: the WireGuard
mailing list on such submissions, so all subscribers here could take a peek on
replies to WG patches even if they don't follow the high-volume netdev list
themselves.

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: Reflections on WireGuard Design Goals

[Roman Mamedov]

On Fri, 10 Aug 2018 14:35:14 +0100
Brian Candler  wrote:

>  From my point of view, the only thing which makes me uncomfortable 
> about wireguard is the lack of any second authentication factor. Your 
> private key is embedded in a plaintext file in your device (e.g. 
> laptop), not even protected with a passphrase.  Anyone who gains access 
> to that laptop is able to establish wireguard connections.
> 
> Of course, it can be argued that the laptop holds other information 
> which is more valuable that the wireguard key, therefore you should 
> concentrate on properly securing the laptop itself (*). Furthermore, to 
> be able to talk to the wireguard kernel module you're already root, and 
> therefore have all sorts of malicious options available to you. etc etc
> 
> But I'd feel a lot happier if a second level of authentication were 
> required to establish a wireguard connection, if no packets had been 
> flowing for more than a configurable amount of time - say, an hour. It 
> would give some comfort around lost/stolen devices.

Couldn't you just encrypt your home directory? Or even the root FS entirely.
Either of those should be a must on a portable device storing valuable
information.

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: Fragmentation on UDP layer possible?

[Roman Mamedov]

On Mon, 13 Aug 2018 02:53:44 +1000
StarBrilliant  wrote:

> I know Wireguard can already do IP layer fragmentation. (Just set
> tunnel MTU >= 1441 then fragmentation will be turned on)

Is that really expected to work? I tried setting MTU 9000 on both ends of a WG
tunnel, but large packets still do not seem to come through properly. Did you
try using it like that in any kind of environment (aside from that one
restrictive network)?

In theory using MTU 9000 or such would help lower the huge overhead percentage
of running IP over VXLAN over IP over WG over IP. I was looking into that the
other day, but my idea was to fragment VXLAN packets across multiple WG ones,
which turned out to be impossible (VXLAN RFC forbids fragmentation).

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Getting IPv6 route advertisements to work over WG

[Roman Mamedov]

Hello,

I am trying to get IPv6 link-local IPs and route advertisements to work over
WG. The reason is not for the usual case of address autoconfiguration, but to
use RA as a dynamic routing protocol of sorts, as it can distribute routes --
or in case of WG (where routes need to be static in AllowedIPs), act as a
keep-alive protocol.

Example use: a host can be connected to a network via a number of independent
routers (and separate WG tunnel to each); in case one of the routers goes
down, the route entry that it was sending via RA times out, so the host will
automatically use the other one(s) to reach that network. It would look
similar to this:

# ip -6 route
...
fd00::/32 via fe80::be:a0ff:fe18:4aac dev wg1 proto ra metric 1024  expires 
30sec pref medium
fd00::/32 via fe80::e8:4fff:fe94:2d7f dev wg2 proto ra metric 1024  expires 
119sec pref medium
fd00::/32 via fe80::43:31ff:fec0:da97 dev wg3 proto ra metric 1024  expires 
86360sec pref low
...

What works:

  * manually assigning link-local(LL) IPs on both sides of a WG tunnel
(fe80:[somethingrandom]/64 scope link);
  * any normal communication over these LL IPs (assuming they are also present
in AllowedIPs);
  * running RADVD with WG link as one of its interfaces;
  * explicitly requesting and receiving a RA, via using 'rdisc6' while 
specifying the
other side's LL IP;

What doesn't:

  * it appears multicast not supported, so anything involving
multicast, as in automatically requesting RAs on the kernel side, or
manually with 'rdisc6' but without specifying peer's LL:

  # rdisc6 wg3
  Soliciting ff02::2 (ff02::2) on wg3...
  Sending ICMPv6 packet: Required key not available

I found discussion[1], but it is unclear what is the outcome. In any case, I
would like to add my vote to please add some kind of multicast support, even
if just as a dumb broadcast for now. It would work just fine for a lot of
cases; don't know about others, but my WG networks tend to include at most 2-3
hosts each (but there's a lot of independent networks).

[1] https://lists.zx2c4.com/pipermail/wireguard/2017-April/001177.html

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: Getting IPv6 route advertisements to work over WG

[Roman Mamedov]

On Mon, 27 Aug 2018 15:32:49 +0200
netrav...@gmail.com wrote:

> When using multicast over WireGuard, would it not be more viable to use
> an extra encapsulation layer to run multicast inside of?
> 
> I am specifically thinking of running either GRE or L2TPv3 over wgX.

I know people run VXLAN or other L2 tunneling protocols over WG. I suppose
you can call that "viable" as in "it can work", but it's a horrible workaround
for the lack of better solution, nothing more. For instance the overhead
reaches comical levels:

  TCP
over IP
  over Ethernet
over VXLAN
  over UDP
over IP
  over Wireguard
over UDP
  over IP 
over Ethernet

Add more fun if you use something else such as PPPoE for Internet connection,
or a 6in4 tunnel for IPv6. At some point the whole thing will break down
because you can no longer fit 1280-byte packets into innermost MTU, and IPv6
won't work.

Not to mention the additional management overhead of an inner L2 tunneling
layer.

Now, if WG would support L2 mode natively (say, with AllowedMACs instead of
AllowedIPs) it would be awesome and that would solve a great number of other
issues as well. But since that appears to be unlikely, and since RAs already
mostly work, with just one piece missing, I hope at least that piece gets
dropped in at some point, and that we aren't stuck at least for this use case
with "more viable" tunneling workarounds forever.

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Error building for ARM

[Roman Mamedov]

Hello,

  AS [M]  net/wireguard/crypto/zinc/curve25519/curve25519-arm.o
net/wireguard/crypto/zinc/curve25519/curve25519-arm.S: Assembler messages:
net/wireguard/crypto/zinc/curve25519/curve25519-arm.S:21: Error: r13 not 
allowed here -- `and sp,sp,#0xfff0'
scripts/Makefile.build:429: recipe for target 
'net/wireguard/crypto/zinc/curve25519/curve25519-arm.o' failed
make[3]: *** [net/wireguard/crypto/zinc/curve25519/curve25519-arm.o] Error 1
scripts/Makefile.build:587: recipe for target 'net/wireguard' failed
make[2]: *** [net/wireguard] Error 2


I guess this could matter?

$ arm-linux-gnueabihf-as --version
GNU assembler (GNU Binutils for Debian) 2.31.1
Copyright (C) 2018 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or later.
This program has absolutely no warranty.
This assembler was configured for a target of `arm-linux-gnueabihf'.

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: [ANNOUNCE] WireGuard Snapshot `0.0.20180708` Available

[Roman Mamedov]

On Sun, 08 Jul 2018 18:52:32 +0200
"Jason A. Donenfeld"  wrote:

>   * receive: use NAPI on the receive path
>   
>   This is a big change that should both improve preemption latency (by not
>   disabling it unconditionally) and vastly improve rx performance on most
>   systems by using NAPI. The main purpose of this snapshot is to test out this
>   technique.

Just ran a few tests, it appears the performance is about 5-10% higher.
Great work!

Two VMs running on the same host (non-WG iperf3 is 14 Gbit/sec), typical
results (upgrading receiver machine only):

Single core:

Before: 1.06 Gbit/sec
After: 1.13 Gbit/sec

Dual core:

Before: 1.25 Gbit/sec
After: 1.35 Gbit/sec

Note that my "before" is a bit non-stock, but with a patch which removed two
calls of "simd_relax" (as I wanted to keep max performance over the
interactivity changes). "After" is the new snapshot without any patches.

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: [ANNOUNCE] WireGuard Snapshot `0.0.20180708` Available

[Roman Mamedov]

On Tue, 10 Jul 2018 16:57:14 +0200
"Jason A. Donenfeld"  wrote:

> The latest snapshot will still have the same preemption relaxation
> with simd_relax(), but gets performance gains by moving to napi, so
> it's still faster overall. If you want the simd_relax() to not take a
> hit and get maximum throughput, the right way of doing this is
> actually to just disable preemption in your kernel with
> PREEMPT_NONE=y.

I build a single kernel to use across a diverse park of machines, including
servers, routers -- and a few GUI desktops. It is not an option for me to
disable preemption entirely in that kernel. (And it would be a hassle to build
two or more kernels each time).

However those of my hosts which are routers with WG, are NOT the same hosts
which are interactive desktops. So I don't want any sacrifices towards
interactivity *in WG*.

I'll probably test again without simd_relax, but from the past mailing list
discussion it seemed like that one doesn't affect things much. In any case,
it's great that you found a way to keep performance and increase interactivity
at the same time with NAPI.

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: [ANNOUNCE] WireGuard Snapshot `0.0.20180708` Available

[Roman Mamedov]

On Tue, 10 Jul 2018 20:57:29 +0500
Roman Mamedov  wrote:

> I'll probably test again without simd_relax

Somehow it's now noticeably worse without those. Even got some dips below
1 Gbit/s which I have never seen before, and the overall speed is lower.

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: [ANNOUNCE] WireGuard Snapshot `0.0.20180708` Available

[Roman Mamedov]

On Tue, 10 Jul 2018 20:38:24 +0200
"Jason A. Donenfeld"  wrote:

> I might not be understanding you correctly. Do you mean to suggest
> that removing simd_relax() actually harms performance now? That having
> it in there helps performance?

Actually no, after your message I swapped kernels again to recheck, and nope,
now the one with simd_relax removed appears faster a bit as it should be (by
about 5%).

Perhaps it was something else, maybe my test bench is not ideal: both
"dual-core" VMs run on the same 8-core FX-8350, which has some of its cores
coupled to share resources, so at the scheduler's whim VMs can probably affect
each other. (Will try further tests with affinity pinning).

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: receive: use gro call instead of plain call

[Roman Mamedov]

On Fri, 13 Jul 2018 08:49:45 -0500
Lonnie Abelbeck  wrote:

> For certain lower-end x86 boxes I test, I noticed WG 0.0.20180708 w/NAPI 
> actually slowed down receive performance.
> 
> Jason recently added "receive: use gro call instead of plain call" [1] 
> commit, which made a big performance improvement.

Yes I'm also seeing about 20% higher performance with this patch (from 1.3-1.4
to 1.6 Gbit on same-host VMs). This is awesome!

...and... if I switch TCP Congestion Control from bbr to illinois on sender, I
now get 2.0 Gbit. WTF. :)

Lonnie, which one do you use on your hosts? Try with illinois (it was the best
choice among all of them from my tests before bbr) and bbr (this one tends to
get a bit lower speeds overall, but ramps up so much faster at the start).

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Include directive to support "conf.d/*" and the like

[Roman Mamedov]

Hello,

I would like to be able to split the [Interface] and [Peer] parts of the config
file into separate files. The reason is that currently I manage configurations
of my various hosts at a central location, then push out common configs to all
hosts.

This becomes problematic with current WireGuard, as it stores both the
host-specific part, and the part common to the entire network, in the same
single file.

While it would be nice if WireGuard had a "hosts/" directory like Tinc uses
(basically storing its equivalents of WG's [Peer] sections each in a separate
file), I feel the most flexible way to support such scenarios would be to have
a generic "Include" directive. That way I could do
"Include /etc/wireguard/peers/*.conf" and then not only store each peer
information in its own file, but also roll-out or fetch and
add/remove/overwrite those files from a central repository.

Also distros could use it by default to enable the often-used "conf.d/*"
mechanism.

Is there anything planned along these lines? Is there a workaround that I
could use with WG in its today's form?

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: Mixed MTU hosts on a network

[Roman Mamedov]

On Fri, 16 Mar 2018 10:35:18 +0100
Matthias Ordner  wrote:

> If you only care about TCP connections you could set a different TCP-MSS 
> with an iptables rule.

On Fri, 16 Mar 2018 11:01:51 +0100
Kalin KOZHUHAROV  wrote:

> You may need to pre-shape the packets for the "offenders", e.g.
> 
> ip6tables -t mangle -A POSTROUTING -o wg0 -d WHATEVERHOST -p tcp -m
> tcp --tcp-flags SYN,RST SYN -j TCPMSS --set-mss 1352
> 
> https://www.netfilter.org/documentation/HOWTO/netfilter-extensions-HOWTO-4.html#ss4.7
> 
> O, wait! You talk IPv6...
> 
> ip6tables -t mangle -A POSTROUTING -o wg0 -d fd39:30::250/128 -p tcp
> -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --set-mss 1372

I knew about this option, but wanted to avoid it because it would incur more
overhead (going to iptables for this) and a bit more complexity.

But guess what, turns out that didn't work either. Tried both OUTPUT and
POSTROUTING chains on the "mangle" table, and set-mss all the way down to
1220, no matter what, the iperf3 output looked the same as before. At this
point I thought I'm going crazy or something. :)

It's not just iperf either, trying to send a file with "netcat6" into a
running listener on the other side also failed to transfer data.

Then almost by accident, I discovered that what also helps. It's to reduce
interface MTU only on the receiver, but just by a bit more, to 1408.

So what makes it work is EITHER:

a) set MTU 1412 on wg0 at sender;

OR

b) set MTU 1408 on wg0 at receiver.

...doing both at the same time is not even necessary. Some tcpdumps from the
receiver host are attached to demonstrate (if anyone else thinks I am crazy :).

Now, I can live with just the impacted (PPPoE) hosts having a lower MTU on wg0.

But still the whole thing seems rather weird.

-- 
With respect,
Roman
Receiver mtu 1420, sender mtu 1412, successful transfer:

# tcpdump -i wg0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on wg0, link-type RAW (Raw IP), capture size 262144 bytes
15:42:35.027995 IP6 fd39:30::2.42414 > fd39:30::250.5001: Flags [S], seq 
4148302601, win 27040, options [mss 1352,sackOK,TS val 2239613851 ecr 
0,nop,wscale 9], length 0
15:42:35.028026 IP6 fd39:30::250.5001 > fd39:30::2.42414: Flags [S.], seq 
505975510, ack 4148302602, win 26960, options [mss 1360,sackOK,TS val 
1473426057 ecr 2239613851,nop,wscale 9], length 0
15:42:35.102517 IP6 fd39:30::2.42414 > fd39:30::250.5001: Flags [.], ack 1, win 
53, options [nop,nop,TS val 2239613925 ecr 1473426057], length 0
15:42:35.102772 IP6 fd39:30::2.42414 > fd39:30::250.5001: Flags [.], seq 
1:1341, ack 1, win 53, options [nop,nop,TS val 2239613925 ecr 1473426057], 
length 1340
15:42:35.102785 IP6 fd39:30::250.5001 > fd39:30::2.42414: Flags [.], ack 1341, 
win 58, options [nop,nop,TS val 1473426131 ecr 2239613925], length 0
15:42:35.102810 IP6 fd39:30::2.42414 > fd39:30::250.5001: Flags [P.], seq 
1341:2145, ack 1, win 53, options [nop,nop,TS val 2239613925 ecr 1473426057], 
length 804
15:42:35.102818 IP6 fd39:30::250.5001 > fd39:30::2.42414: Flags [.], ack 2145, 
win 64, options [nop,nop,TS val 1473426131 ecr 2239613925], length 0
15:42:35.729846 IP6 fd39:30::250.5001 > fd39:30::2.42162: Flags [F.], seq 
1811803733, ack 3749581328, win 56, options [nop,nop,TS val 1473426758 ecr 
2239251660,nop,nop,sack 1 {1341:2145}], length 0
15:42:35.804023 IP6 fd39:30::2.42162 > fd39:30::250.5001: Flags [.], ack 1, win 
54, options [nop,nop,TS val 2239614627 ecr 1473426758,nop,nop,sack 1 {0:1}], 
length 0
15:42:36.939584 IP6 fd39:30::2.42414 > fd39:30::250.5001: Flags [F.], seq 2145, 
ack 1, win 53, options [nop,nop,TS val 2239615763 ecr 1473426131], length 0
15:42:36.939723 IP6 fd39:30::250.5001 > fd39:30::2.42414: Flags [F.], seq 1, 
ack 2146, win 64, options [nop,nop,TS val 1473427968 ecr 2239615763], length 0
15:42:37.014143 IP6 fd39:30::2.42414 > fd39:30::250.5001: Flags [.], ack 2, win 
53, options [nop,nop,TS val 2239615837 ecr 1473427968], length 0
^C
12 packets captured
12 packets received by filter
0 packets dropped by kernel

===

Receiver mtu 1408, sender mtu 1420, successful transfer:

# tcpdump -i wg0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on wg0, link-type RAW (Raw IP), capture size 262144 bytes
15:43:23.935508 IP6 fd39:30::2.42442 > fd39:30::250.5001: Flags [S], seq 
1011924297, win 27200, options [mss 1360,sackOK,TS val 2239662759 ecr 
0,nop,wscale 9], length 0
15:43:23.935541 IP6 fd39:30::250.5001 > fd39:30::2.42442: Flags [S.], seq 
1735470303, ack 1011924298, win 26720, options [mss 1348,sackOK,TS val 
1473474964 ecr 2239662759,nop,wscale 9], length 0
15:43:24.009867 IP6 fd39:30::2.42442 > fd39:30::250.5001: Flags [.], ack 1, win 
54, options [nop,nop,TS val 2239662834 ecr 1473474964], length 0
15:43:24.010192 IP6 fd39:30::2.42442 > fd39:30::250.5001: Flags [.], seq 
1:1337, ack 1, win 54, options 

Mixed MTU hosts on a network

[Roman Mamedov]

Hello,

I have a host which is on PPPoE and has 1492 as underlying MTU.

When WireGuard starts by default, it sets MTU of its interface to 1420. All
TCP connections trying to send a stream of data over the WG interface to that
host, hang up (I test with iperf3).

My first idea was to override the MTU for this specific host via adding a
route:

# ip -6 route add fd39:30::250/128 dev wg0 mtu 1412 metric 1

# ip -6 route | grep ^fd39:30
fd39:30::250 dev wg0  metric 1  mtu 1412
fd39:30::/64 dev wg0  proto kernel  metric 256

# ip route get fd39:30::250
fd39:30::250 from :: dev wg0  src fd39:30::2  metric 1  mtu 1412

However, this does not help at all. Even adding the corresponding route on the
other side. Even using the "mtu lock" keyword instead of just "mtu". I am still
puzzled why. Any ideas?

=
# iperf3 -c fd39:30::250
Connecting to host fd39:30::250, port 5201
[  4] local fd39:30::2 port 44902 connected to fd39:30::250 port 5201
[ ID] Interval   Transfer Bandwidth   Retr  Cwnd
[  4]   0.00-1.00   sec   474 KBytes  3.88 Mbits/sec1   1.31 KBytes   
[  4]   1.00-2.00   sec  0.00 Bytes  0.00 bits/sec1   1.31 KBytes   
[  4]   2.00-3.00   sec  0.00 Bytes  0.00 bits/sec1   1.31 KBytes   
[  4]   3.00-4.00   sec  0.00 Bytes  0.00 bits/sec0   1.31 KBytes   
[  4]   4.00-5.00   sec  0.00 Bytes  0.00 bits/sec1   1.31 KBytes   
[  4]   5.00-6.00   sec  0.00 Bytes  0.00 bits/sec0   1.31 KBytes   
[  4]   6.00-7.00   sec  0.00 Bytes  0.00 bits/sec0   1.31 KBytes   
[  4]   7.00-8.00   sec  0.00 Bytes  0.00 bits/sec0   1.31 KBytes   
[  4]   8.00-9.00   sec  0.00 Bytes  0.00 bits/sec0   1.31 KBytes   
[  4]   9.00-10.00  sec  0.00 Bytes  0.00 bits/sec1   1.31 KBytes   
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval   Transfer Bandwidth   Retr
[  4]   0.00-10.00  sec   474 KBytes   388 Kbits/sec5 sender
[  4]   0.00-10.00  sec  0.00 Bytes  0.00 bits/sec  receiver

iperf Done.
=

What helps, is only reducing MTU of the entire wg0 interface to 1412. Then
everything works fine. But it doesn't feel optimal to reduce MTU of the entire
network just because of 1 or 2 hosts. I would rather use a couple of those
mtu-override routes, if they worked.

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: Include directive to support "conf.d/*" and the like

[Roman Mamedov]

On Sat, 14 Apr 2018 03:47:57 +0200
"Jason A. Donenfeld"  wrote:

> Hi Roman,
> 
> This also came up in another thread I was replying to earlier tonight.
> While one way indeed is to have an 'include' directive, it seems
> simple enough to just do something like:
> 
> $ wg setconf wg0 <(cat /etc/wireguard/mysite.conf.d/*.conf)
> 
> And then you can have various fragments in there like:
> 
> 000-interface.conf
> 001-peergroupA.conf
> 001-peergroupB.conf
> 001-peergroupC.conf
> 
> And so forth. Would this be an acceptable solution for you?

Yeah, thanks. I settled on a solution similar to this. Since WG in my case is
"external" to the main OS (i.e. not wired into standard initscripts or network
configuration), I have my own shell-script bringing it up anyways -- and that
script might as well pre-process or generate the configuration file. So now I
build a full config file in /tmp/ from various pieces and auto-detected
host-specific conditions, and then do a setconf to that. (Rather than addconf
as some suggested, I prefer to have the complete file available on disk for
inspection in case any debugging is needed).

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: Mixed MTU hosts on a network

[Roman Mamedov]

On Sat, 14 Apr 2018 15:16:56 +0200
"Jason A. Donenfeld"  wrote:

> Hi Roman,
> 
> This commit should fix it. It now has a unit test too so that we don't
> hit this issue again. Thanks for reporting it in such detail.
> 
> https://git.zx2c4.com/WireGuard/commit/?id=a88a067d5477f877003d3703bb3b95cb4e94bc46
> 
> Let me know if that fixes it on your end.
> 
> Jason

Thanks! I didn't get a chance to test it yet.

Leaving route MTUs aside, did you look into why the interface MTU of 1412
behaves erratically (while by all calculations it should just fit into 1492
underlying PPPoE MTU), with only 1408 working reliably? Is it also because of
the padding?

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: Mixed MTU hosts on a network

[Roman Mamedov]

On Sat, 14 Apr 2018 16:15:07 +0200
"Jason A. Donenfeld"  wrote:

> Hi Roman,
> 
> I answered this in my first email to you, which perhaps got lost in
> the mix of emails, so I'll quote the relevant part:
> 
> > 2) When we pad the packet payload. In this case, we pad it to the
> > nearest multiple of 16, but we don't let it exceed the device MTU.
> > This is skb_padding in send.c. This behavior seems like the bug in
> > your particular case, since what matters here is the route's MTU, not
> > the device MTU. For full 1412 size packets, the payload is presumably
> > being padded to 1424, since that's still less than the device MTU. In
> > order to test this theory, try setting your route MTU, as you've
> > described in your first email, to 1408 (which is a multiple of 16). If
> > this works, let me know, as it will be good motivation for fixing
> > skb_padding. If not, then it means there's a problem elsewhere to
> > investigate too.
> 
> In short, because 1408 is a multiple of 16 so it didn't get rounded
> up, whereas 1412 got rounded up to 1424.

I got that, but that still seemed to be talking about the problem with route
MTUs.

But what about if I don't touch any route MTUs at all, but set the WG device
MTU to 1412. In my further experiments that didn't work well either, causing
weird one-directional issues, and only 1408 worked.

So, is it possible to fix the padding so 1412 can be used as WG device MTU on
underlying MTU of 1492? Otherwise, shouldn't there be a warning somewhere in
the docs to not just choose the largest fitting MTU according to [1], but also
round down what you got, to a nearest multiple of 16.

[1] https://www.mail-archive.com/wireguard@lists.zx2c4.com/msg01856.html

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: Mixed MTU hosts on a network

[Roman Mamedov]

On Sat, 14 Apr 2018 16:45:32 +0200
"Jason A. Donenfeld"  wrote:

> In this case, WireGuard seems to be doing the right thing. Think you
> could come up with some minimal test that exhibits the behavior you're
> seeing?

I now remember in more detail what was the problem. It was not with MTU 1412
on both sides, it was during trying to mix WG MTU 1412 on the PPPoE-connected
machine, with WG MTU 1420 on the other side (which uses full 1500 underlying
MTU).

Here I posted about it with some tcpdumps included:
https://lists.zx2c4.com/pipermail/wireguard/2018-March/002537.html

With 1420 on the "full MTU" side, the "PPPoE" side had to set 1408 WG MTU for
things to work properly, not 1412 as would theoretically fit into its PPPoE.

I'll post an update if I come up with a short and simple reproducer sequence.

Setting 1412 on both sides seems to work fine from more testing just now.

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: Why does 'allowed-ips' affect route selection behavior?

[Roman Mamedov]

On Sun, 15 Apr 2018 14:49:23 -0400
"Patrick O'Sullivan"  wrote:

> $ sudo ip route get 4.2.2.1
> 4.2.2.1 dev wg0 table 51820 src 10.111.111.100
  ^^^
> cache

> Can someone please explain this behavior?

Probably will be easier to do if you show the output of "ip -4 rule".

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: add/remove a peer

[Roman Mamedov]

On Sun, 25 Mar 2018 21:17:35 +0200
Kalin KOZHUHAROV  wrote:

> There is a reason, at least one, good one - it is called simplicity.
> It is also hard to work when you are running out of disk space or
> memory; do you expect WG to solve that for you?
> Simply put, IP addressing schemes are not a part of WG, neither a requirement.
> There are many ways to use WG and "assign random, free IP address and
> send to a new peer" is too specific of a use case.
> 
> May be you can cobble up something with a DHCP server that cares about
> certain address range?
> Or a simple flat-file dB and a script that does it for you?
> 
> What happens when you run out of addresses?
> How do you re-assign an IP address to a new peer?
> ...
> Those are questions widely outside WG, IMHO.

Agreed.

One more idea that comes to mind, is to use IPv6 and assign IPs based on peer
public keys. Assuming a fixed /64 subnet, using a 64-bit half of the public
key for the host part, still makes collisions nearly impossible.

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Reconciling "cryptokey-based" and regular routing

[Roman Mamedov]

Hello,

I need to have multiple gateways on my WG network that can provide access to
the entire IPv4 (or IPv6) Internet, for redundancy and load-balancing
purposes.

In WG terms this means I need to set AllowedIPs to 0.0.0.0/0 on more than one
peer. Then I would add routes into the regular routing table for various
destinations,

ip -4 route add 8.8.8.8 via 10.0.0.1
ip -4 route add 8.8.4.4 via 10.0.0.2

or

ip -4 route add default \
  nexthop via 10.0.0.1 weight 1 \
  nexthop via 10.0.0.2 weight 1

or whatever.

But as documentation and some testing show, this can't really work in WG's
"cryptokey-routing" system. If multiple hosts have 0.0.0.0/0 as allowed IPs,
WG just sends everything to a random one of them (the first one?),
disregarding all of the routing table settings from the examples above.

Is there any possibility to still use multiple routers like that?

If not, then could you add an option to not use AllowedIPs for routing?
Or at least to not enforce filtering on incoming packets -- then perhaps I
could have only 10.0.0.1 and 10.0.0.2 in AllowedIPs for those hosts, and
outgoing routing would work properly, with replies from Internet hosts not
getting filtered out?

(Apologies for multiple posts per day, I'm just deploying WireGuard for the
first time today, and it's quite unusual compared to what I used before. I
will stop soon :)

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: Mixed MTU hosts on a network

[Roman Mamedov]

On Fri, 16 Mar 2018 15:53:43 +0500
Roman Mamedov <r...@romanrm.net> wrote:

> But guess what, turns out that didn't work either. Tried both OUTPUT and
> POSTROUTING chains on the "mangle" table, and set-mss all the way down to
> 1220, no matter what, the iperf3 output looked the same as before.

Actually the iptables bit is easy to explain. Even if initial MSS is forced
to a low value on the sender, it's get negotiated back up to the maximum value
according to MTU on the receiver (changed both IPs since then):

21:13:38.641531 IP6 fd39:30::f5a8:e923:f8cd:24b5.40052 > 
fd39:30::e84f:942d:7f93:ddc1.5001: Flags [S], seq 2397878391, win 27200, 
options [mss 1220,sackOK,TS val 566161815 ecr 0,nop,wscale 9], length 0
21:13:38.641574 IP6 fd39:30::e84f:942d:7f93:ddc1.5001 > 
fd39:30::f5a8:e923:f8cd:24b5.40052: Flags [S.], seq 1221117548, ack 2397878392, 
win 26800, options [mss 1352,sackOK,TS val 2726162536 ecr 566161815,nop,wscale 
9], length 0
21:13:38.716047 IP6 fd39:30::f5a8:e923:f8cd:24b5.40052 > 
fd39:30::e84f:942d:7f93:ddc1.5001: Flags [.], ack 1, win 54, options 
[nop,nop,TS val 566161889 ecr 2726162536], length 0
21:13:38.716444 IP6 fd39:30::f5a8:e923:f8cd:24b5.40052 > 
fd39:30::e84f:942d:7f93:ddc1.5001: Flags [P.], seq 1341:1605, ack 1, win 54, 
options [nop,nop,TS val 566161889 ecr 2726162536], length 264
21:13:38.716458 IP6 fd39:30::e84f:942d:7f93:ddc1.5001 > 
fd39:30::f5a8:e923:f8cd:24b5.40052: Flags [.], ack 1, win 55, options 
[nop,nop,TS val 2726162611 ecr 566161889,nop,nop,sack 1 {1341:1605}], length 0

So the other side really needs to have a proper MTU set. And the highest working
wg0 MTU on PPPoE turned out to be 1408, not 1412 as I assumed. As for why 1412
also works but only if set on the sender side, I've no explanation for that yet.

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: Sending just ssh traffic via wg

[Roman Mamedov]

On Sat, 6 Oct 2018 11:21:01 +0100
Brian Candler  wrote:

> (Aside: I wish ssh had a feature like SNI, so that you could build an 
> ssh proxy that forwards incoming connections to the right host.  I have 
> done this before using an inbound SOCKS proxy, but it's messy to use)

What insane things people invent only not to use IPv6 :)

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: IPv6 Not Getting Past Server

[Roman Mamedov]

On Sat, 22 Sep 2018 15:55:22 -0400
"Aaron W. Swenson"  wrote:

> I’m going to use the official documentation IP addresses. I am using real IPv6
> addresses and not using NAT66. Naturally, NAT is being used for IPv4. Here are
> the definitions I’m using:
> 
> Server Public IPv6: 2001:DB8::DEAD:F00D/64
> Server Public IPv4: 192.0.2.1
> Routed /116: 2001:DB8::BEEF:3000/116

Are you sure you have a *routed* subnet from your ISP? Moreover, does
this /116 lie within the above /64? (in your obfuscated example it does)

It could be that your provider gives you not a routed subnet, but an on-link
one instead. In which case you can use IPs from it on your server directly,
but can't route it somewhere else (because the upstream router expects all IPs
to be reachable directly on the same link that it has to your server).

One workaround is to set up ndppd, which turns an on-link subnet into routed:
https://github.com/DanielAdolfsson/ndppd
But in my experience it does not work on all ISPs/hosts.


> Server Wireguard IPv6: 2001:DB8::BEEF:3001
> Server Wireguard IPv4: 10.0.0.1
> Client Wireguard IPv6: 2001:DB8::BEEF:3002
> Client Wireguard IPv4: 10.0.0.2

You didn't post your WireGuard's AllowedIPs setting, but I tend to assume
everything is fine there. Just don't forget that to route to the outside world
AllowedIPs on the client must contain ::/0 for the server.

> I can ping the outside world through IPv4 just fine. However, with IPv6 I can
> only ping the server’s IPv6 addresses (2001:DB8::BEEF:3001 and
> 2001:DB8::DEAD:F00D). The outside world stays out of reach. The packets are 
> just
> dropped. I’m not getting network unreachable or any other error message back.

Run tcpdump on the server both on the WG side and on the outgoing interface,
and you'll be able to say more precisely where or by whom they are dropped.

> When I enabled forwarding for IPv6 on the server, I did have to manually add
> the route so that IPv6 would continue working on the server
> (ip -r route add default fe80::1).

This is because the default behavior is such that enabling routing precludes
accepting Route Advertisements from upstream routers, so you lost the route
you were getting via those. To keep accepting them, set accept_ra to 2
(echo 2 > /proc/sys/net/ipv6/conf/ethX/accept_ra)

> I can SSH into the server, and ping the
> outside world no problem. And, the outside world can reach my server via IPv6
> just fine, too.

Try adding one of the IPs you wanted to route into WG directly to server's
uplink interface and see if it becomes pingable from the outside world. If so,
that would confirm (if a little bit) the non-routed subnet hypothesis.

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: mesh VPN with wireguard?

[Roman Mamedov]

On Thu, 28 Mar 2019 23:22:45 +0900
Tomasz Chmielewski  wrote:

> Does Wireguard allow to set up mesh VPN with "relative ease"?
> 
> Say, we have 10 servers with public IPs, we want them all to create a 
> VPN network with private subnet 10.11.12.0/24, and have all 10 servers 
> communicate directly with each other.
> Then a year later, expand it to 100 servers.

Sure.

But note that in this case unlike Tinc you cannot have some servers exit to
the outside world via some other servers (with AllowedIP 0.0.0.0/0). There has
to be just one such exit point per a WG network.

If it's purely for communication between servers, then of course no issue.

> Something in the line of: https://www.tinc-vpn.org/

Another limitation compared to Tinc is that Tinc will autoheal the partially
disconnected mesh and will have some nodes forwarding for the others, in case
direct communication between some of them gets cut (e.g. due to a peering or
routing issue on the underlying Internet -- this saved me a few times).

WG will do no such thing, and node-to-node communication working will depend
on both nodes always having direct connectivity to each other.

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: Help calculate MTU, ISP's 1448

[Roman Mamedov]

On Tue, 26 Feb 2019 12:39:50 +
"STR ."  wrote:

> I have Fiber to our apartment complex basement, from there Cat6 runs to
> each apartment. The ISP/apartment service provider suggests an MTU of
> 1448, which I set for the PPPoE interface on my OpenWRT router.

It could be that your ISP meant this as the pre-PPPoE MTU, so 1448 on ethX
link, but 1440 inside PPPoE.

> I read 
> https://lists.zx2c4.com/pipermail/wireguard/2017-December/002201.html
> which comes to (assuming 1500 byte MTU) to 60 bytes (IPv6) to 80 bytes less 
> to account for Wireguard protocol overhead.
> 
> Using this info, I tried an MTU of both (1448-80=1368) and (1448-
> 60=1388).
> As my ISP assigns only IPv4, I expected an MTU of 1388 to work, which I
> set on the Wireguard interface in OpenWRT.
> 
> However, when set to 1388, almost everything works except any Google
> related sites like Maps, Gmail, YT etc.
> When set to 1368, everything works and it's the way I have it setup
> right now.

Try 1380 for a bit.

> What am I missing here?
> Why won't Google sites load via my WG VPN when the MTU is set to 1388?
> 
> If it helps, I host the WG server on Google's cloud platform and was
> informed that GCP has an MTU of 1460 bytes.

Are you setting the same (lower of the two maximums) MTU on both ends of the
wireguard tunnel? (You should)

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

ifconfig lists IPv6 twice for one WG interface

[Roman Mamedov]

Hello,

I'm facing a strange issue where "ifconfig" shows the IPv6 twice for one
particular WG interface. Other similar interfaces on the same machine aren't
affected. Can't pinpoint what's special about this one yet.

The IP is not added twice during interface setup. Adding it once more, as
expected results in

  # ip -6 addr add fd39:aa:3dc2:a78a:7900:fcd:12a3:6181/64 dev wgw
  RTNETLINK answers: File exists

The "ip" tool doesn't list the address twice:

  # ip a l dev wgw
  44: wgw:  mtu 1432 qdisc noqueue state UNKNOWN 
group default qlen 1000
  link/none 
  inet6 fd39:aa:3dc2:a78a:7900:fcd:12a3:6181/64 scope global 
 valid_lft forever preferred_lft forever

But please no comments about "ifconfig" being deprecated, I prefer to keep using
it for certain tasks.

Now adding some debug print to my interface creation script, with:

  ifconfig $IFACE
  ip -6 addr add $MYIP/64 dev $IFACE || exit 1
  ifconfig $IFACE

This results in such output:

  wgw   Link encap:UNSPEC  HWaddr 
00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  
POINTOPOINT NOARP  MTU:1420  Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000 
RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

  wgw   Link encap:UNSPEC  HWaddr 
00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  
inet6 addr: fd39:aa:3dc2:a78a:7900:fcd:12a3:6181/64 Scope:Global
inet6 addr: fd39:aa:3dc2:a78a:7900:fcd:12a3:6181/64 Scope:Global
POINTOPOINT NOARP  MTU:1420  Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000 
RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

Overall this doesn't seem to break anything, but might be a sign of something 
not gong
entirely right under the hood. Anyone seen the same thing?

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: Wireguard fails to start when adding IPv6 to AllowedIP

[Roman Mamedov]

On Sun, 03 Mar 2019 08:56:12 +0100
XRP  wrote:

> [#] ip link set mtu 1200 up dev wg1
> [#] ip route add fdb8:a70c:b109:9935::/64 dev wg1
> RTNETLINK answers: No such device

IPv6 cannot work with MTU less than 1280 on the device.

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: Logical cores / SMT with WireGuard

[Roman Mamedov]

On Thu, 14 Feb 2019 18:02:26 +
Lee Yates  wrote:

Sorry, hit "send" before reading the rest of your message.

> the router runs headless and is awkward to get a monitor to so I can access
> the BIOS.

You can toggle it without needing the BIOS.
It is possible to disable SMT from grub, with Linux kernel boot arguments.
It even seems possible to disable/enable it without a reboot.
See https://www.golinuxhub.com/2018/01/how-to-disable-or-enable-hyper.html

> My WAN is 'only' 400Mbps anyway so
> hardly a taxing test. Because of this, I can't really learn about how
> much WireGuard benefits from the extra threads, if it does at all, as
> either way I have headroom to spare for my current WAN provision.

Set up a separate WG network with a peer on your Gbit LAN. Or even run a
virtual machine on the same host, and run WG between host and VM, which should
get you multi-Gbit raw throughput and likely make WG encryption the
bottleneck. That way you can observe not only the CPU load, but also the
transfer speed reached changing with HT on/off.

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: Logical cores / SMT with WireGuard

[Roman Mamedov]

On Thu, 14 Feb 2019 18:02:26 +
Lee Yates  wrote:

> recommendations to disable HT, I got to wondering how much - if at all -
> disabling HT would impact on WireGuard's real world performance. I mean,
> it obviously can utilise logical cores/threads, but is there a real
> world throughput benefit vs just using the real cores?

This sounds like something YOU are in a great position to test, and then write
an interesting blog post or mailing list message summarising the results. :)

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

WG can now be fragmented -- great!

[Roman Mamedov]

Hello,

Just wanted to share my excitement about
https://git.zx2c4.com/WireGuard/diff/?id=57a8ca7f49b5e70aae18b8b5a70cde8f9e4a9346=7cf2dae97635c8c20a8943522bab2b56c6885c8d

This means WG packets can now be fragmented, and as such we can use arbitrary
large MTU inside WG. This in turn means we can now use WG to transport full
9000 MTU VXLAN frames over the Internet:

# ifconfig wg10
wg10  Link encap:UNSPEC  HWaddr 
00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  
  inet6 addr: fd39:aa:6089:5d42:7900:fcd:12a3:6181/64 Scope:Global
  UP POINTOPOINT RUNNING NOARP  MTU:9070  Metric:1
  RX packets:12405 errors:0 dropped:0 overruns:0 frame:0
  TX packets:11130 errors:17 dropped:2 overruns:0 carrier:8
  collisions:0 txqueuelen:1000 
  RX bytes:81966214 (78.1 MiB)  TX bytes:45563644 (43.4 MiB)

# ifconfig xwg10
xwg10 Link encap:Ethernet  HWaddr 02:79:00:0f:cd:12  
  inet addr:10.123.0.250  Bcast:10.123.0.255  Mask:255.255.255.0
  inet6 addr: fe80::79:ff:fe0f:cd12/64 Scope:Link
  UP BROADCAST RUNNING MULTICAST  MTU:9000  Metric:1
  RX packets:12369 errors:0 dropped:0 overruns:0 frame:0
  TX packets:9577 errors:9 dropped:0 overruns:0 carrier:9
  collisions:0 txqueuelen:1000 
  RX bytes:80678848 (76.9 MiB)  TX bytes:44408417 (42.3 MiB)

# ping 10.123.0.1 -s 8972 -M do
PING 10.123.0.1 (10.123.0.1) 8972(9000) bytes of data.
8980 bytes from 10.123.0.1: icmp_seq=1 ttl=64 time=78.7 ms
8980 bytes from 10.123.0.1: icmp_seq=2 ttl=64 time=77.2 ms
8980 bytes from 10.123.0.1: icmp_seq=3 ttl=64 time=82.0 ms
8980 bytes from 10.123.0.1: icmp_seq=4 ttl=64 time=77.5 ms
^C
--- 10.123.0.1 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3003ms
rtt min/avg/max/mdev = 77.214/78.881/82.054/1.940 ms

08:39:47.573368 IP6 rin.romanrm.net > 
dynamic-2a02-2698-8024-0.tmn.ertelecom.ru: frag (0|1440) 710 > 710: UDP, bad 
length 9102 > 1432
08:39:47.573371 IP6 rin.romanrm.net > 
dynamic-2a02-2698-8024-0.tmn.ertelecom.ru: frag (1440|1440)
08:39:47.573374 IP6 rin.romanrm.net > 
dynamic-2a02-2698-8024-0.tmn.ertelecom.ru: frag (2880|1440)
08:39:47.573376 IP6 rin.romanrm.net > 
dynamic-2a02-2698-8024-0.tmn.ertelecom.ru: frag (4320|1440)
08:39:47.573378 IP6 rin.romanrm.net > 
dynamic-2a02-2698-8024-0.tmn.ertelecom.ru: frag (5760|1440)
08:39:47.573380 IP6 rin.romanrm.net > 
dynamic-2a02-2698-8024-0.tmn.ertelecom.ru: frag (7200|1440)
08:39:47.573383 IP6 rin.romanrm.net > 
dynamic-2a02-2698-8024-0.tmn.ertelecom.ru: frag (8640|470)
08:39:48.575079 IP6 dynamic-2a02-2698-8024-0.tmn.ertelecom.ru > 
rin.romanrm.net: frag (0|1440) 710 > 710: UDP, bad length 9102 > 1432
08:39:48.575189 IP6 dynamic-2a02-2698-8024-0.tmn.ertelecom.ru > 
rin.romanrm.net: frag (1440|1440)
08:39:48.575339 IP6 dynamic-2a02-2698-8024-0.tmn.ertelecom.ru > 
rin.romanrm.net: frag (2880|1440)
08:39:48.575448 IP6 dynamic-2a02-2698-8024-0.tmn.ertelecom.ru > 
rin.romanrm.net: frag (4320|1440)
08:39:48.575565 IP6 dynamic-2a02-2698-8024-0.tmn.ertelecom.ru > 
rin.romanrm.net: frag (5760|1440)
08:39:48.575691 IP6 dynamic-2a02-2698-8024-0.tmn.ertelecom.ru > 
rin.romanrm.net: frag (7200|1440)
08:39:48.575693 IP6 dynamic-2a02-2698-8024-0.tmn.ertelecom.ru > 
rin.romanrm.net: frag (8640|470)
08:39:48.575828 IP6 rin.romanrm.net > 
dynamic-2a02-2698-8024-0.tmn.ertelecom.ru: frag (0|1440) 710 > 710: UDP, bad 
length 9102 > 1432
08:39:48.575831 IP6 rin.romanrm.net > 
dynamic-2a02-2698-8024-0.tmn.ertelecom.ru: frag (1440|1440)
08:39:48.575833 IP6 rin.romanrm.net > 
dynamic-2a02-2698-8024-0.tmn.ertelecom.ru: frag (2880|1440)
08:39:48.575834 IP6 rin.romanrm.net > 
dynamic-2a02-2698-8024-0.tmn.ertelecom.ru: frag (4320|1440)
08:39:48.575837 IP6 rin.romanrm.net > 
dynamic-2a02-2698-8024-0.tmn.ertelecom.ru: frag (5760|1440)
08:39:48.575838 IP6 rin.romanrm.net > 
dynamic-2a02-2698-8024-0.tmn.ertelecom.ru: frag (7200|1440)
08:39:48.575840 IP6 rin.romanrm.net > 
dynamic-2a02-2698-8024-0.tmn.ertelecom.ru: frag (8640|470)

I also briefly tested performance and despite fragmentation having a bad
reputation for some, I don't see much difference in iperf speeds to
the same host vs going directly.

This is now usable to join multiple locations via VXLAN interfaces as members
of L2 bridges to physical 1G/10G networks without hobbling MTU of the latter.

Thanks!

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Kernel thread naming

[Roman Mamedov]

Hello,

Today I noticed there are kernel threads named "wg-crypt-wgX" (the latter part
being name of the interface). However when there is actual load on WG, these
don't seem to be active, and in "top" we still see a bunch of "kworker/0:X"
using the CPU.

Would it be possible to give those kworkers recognizable names instead?
I remember Btrfs had a change like this in the past, first they had all
kworkers, and now a whole assortment of threads like "btrfs-cleaner",
"btrfs-transacti" or "btrfs-endio-met" (there's some strict length limit).

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Revisiting the weird MTU issue

[Roman Mamedov]

Hello,

I use WireGuard over IPv6 on a PPPoE connection. The Internet interface MTU is
1492. By my calculations MTU 1412 on the WG interface should fit.

However, the following occurs on various MTU combinations between the Remote
(a server in a DC with full 1500 wire MTU) and Local WG interface MTUs:

Fails or not, is whether a within-WG Remote->Local TCP connection (iperf3)
works fine or hangs up after transferring a few initial bits of data.

Remote  Local  Result

  1420   1420  Fails (as expected)

  1420   1412  Fails (weird)

  1412   1412  Works (fair enough)

  1420   1408  Works (super weird!!!)

Now I hope I described the situation clearer than the last time posting about
this, so maybe someone has an idea what could be the culprit?

So far this doesn't cause too much issue, as I'm using on designated p2p links
for when one of the peers is on this PPPoE, I just use 1412 on both sides. But
still, surely the above shouldn't happen?

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: Speed on Raspberry Pi 4

[Roman Mamedov]

On Sat, 29 Jun 2019 12:38:01 +0200
Christopher Bachner  wrote:

> In htop I can see that one of the 4 cores is running at 99%. So I assume
> that is the bottleneck.
> 
> Is there a way to improve this? I assume it does not matter which side is
> the server and which is the client?

You can see that the load from WireGuard encryption is about 42-43% per each
core. But the thing is, one of them (the 1st) also gets to process interrupt
load from the NIC, and that consumes the rest of it, causing the bottleneck. In
theory, if you could limit WG to run encryption on all cores EXCEPT the first
one, then mybe...

Another way would be to use a NIC which is capable of splitting interrupt load
across multiple CPU cores. But I believe the RPi one can't do that, and no USB
NICs can.

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: Improve "[WireGuard] Header / MTU sizes for Wireguard"

[Roman Mamedov]

On Wed, 17 Jul 2019 17:45:18 +0800
Yousong Zhou  wrote:

> For WireGuard overhead breakdown [1], maybe it's worth also mentioning
> that N the length of encrypted data will be padded to be multiples of
> 16.
> 
> I am only aware of this when fragmentation was spotted.  With 1500 as
> MTU for ethernet, PPPoE has MTU 1492 (1500 - 8).  I thought 1432 (1492
> - 60) for wireguard should work for ipv4-only traffic. It needs to be
> 1424 to avoid fragmentation.

1432 should work as long as you set it on *both* ends of your WireGuard tunnel.
I wrote about this here (expect mine was on IPv6, so all MTUs listed are 20
bytes lower): https://lists.zx2c4.com/pipermail/wireguard/2019-April/004078.html
Could you try 1432 on both endpoints and confirm it works (or not)?

So far I don't know any clear explanation of what's described in the above
referenced message. Also that was before the IPv6 fragmentation was allowed
for WG, so now it will change (likely will still work and send fragmented
packets, instead of all the "Fail" cases in the table).

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: [PATCH] wg-quick: linux: add support for nft and prefer it

[Roman Mamedov]

On Tue, 10 Dec 2019 18:36:06 +0100
"Jason A. Donenfeld"  wrote:

> That bachelors thesis says in the abstract, "Latency was measured
> through the round-trip time of ICMP packets while throughput was
> measured by generating UDP traffic using iPerf3. The results showed
> that, when using linear look-ups, nftables performs worse than
> iptables when using small frame sizes and when using large rulesets.

Smallest possible frame sizes are what matters the most when testing any
router or firewall setup, because only then you will hit the packet-per-second
limits of the actual firewalling/routing engine. Good performance at large
frame sizes is not an impressive achievent, there you will just hit
on-the-wire bandwidth limits sooner than the CPU toll of processing rulesets
or routing lookups for each of those frames will begin to matter.

> On the other hand, if what you say is actually true in our case, and
> nftables is utter crap, then perhaps we should scrap this nft(8) patch
> all together and just keep pure iptables(8). DKG - you seemed to want
> nft(8) support, though. How would you feel about that sort of
> conclusion?

Even with my view of it I do not argue for removing nftables support from
your tools, realistically it's probably not going anywhere, or at least not
soon enough, just thought I should point out that "nftables is faster" should
not be so naturally assumed to be the case, and if I dare to say that
everyone should decide for themselves what tools they prefer, and to carefully
weigh all benefits and downsides of the proposed alternatives -- not just come
along obediently with some external party that "knows what is best for you" and
declares something deprecated out of their own arbitrary reasons.

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: [PATCH] wg-quick: linux: add support for nft and prefer it

[Roman Mamedov]

On Tue, 10 Dec 2019 17:54:49 +0100
"Jason A. Donenfeld"  wrote:

> iptables rules and nftables rules can co-exist just fine, without any
> translation needed. Indeed if your iptables is symlinked to
> iptables-nft, then you'll insert nftables rules when you try to insert
> iptables rules, but it really doesn't matter much either way (AFAIK).
> I figured I'd prefer nftables over iptables if available because I
> presume, without any metrics, that nftables is probably faster and
> slicker or something.

nftables is slower than iptables across pretty much every metric[1][2]. It
only wins where a pathological case is used for the iptables counterpart (e.g.
tons of single IPs as individual rules and without ipset). It is a disaster
that it is purported to be the iptables replacement, just for the syntax and
non-essential whistles such as updating rules in place or something. And
personally I don't prefer the new syntax either. It's the systemd and
pulseaudio story all over again, where something more convoluted, less reliable
and of lower quality is passed for a replacement of stuff that actually worked,
but was deemed "unsexy" and arbitrarly declared as deprecated.

[1] http://www.diva-portal.org/smash/get/diva2:1212650/FULLTEXT01.pdf
[2] https://developers.redhat.com/blog/2017/04/11/benchmarking-nftables/

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: idle traffic considerations

[Roman Mamedov]

On Fri, 29 Nov 2019 16:18:52 -0500
zrm  wrote:

> Ballpark estimate, round a keepalive packet to about a hundred bytes. 
> You're also going to get a re-keys, call those two hundred bytes. If you 
> have a keepalive every 30 seconds and a re-key every 120 seconds, that's 
> around 18KB per hour per peer in each direction.

And read the small-print of mobile carrier plans, at least in our country[1]
they love so much to tally-up the user transferred data every hour, while also
rounding that up to nearest 250 KB, or even 1 MB. So even in the above
scenario they would bill for at least 250 KB/hour.

[1] http://tyumen.megafon.ru/tariffs/all/megafon-online.html

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: wireguard slow pings

[Roman Mamedov]

On Sun, 16 Feb 2020 07:58:48 -0500
Neal Becker  wrote:

> I'm testing wireguard
> wireguard-0.0.20191219-2.fc31.x86_64
> between a Fedora 31 client and server, comparing to openvpn.
> 
> Openvpn is running between a linux client outside my lan and a server on my
> router, which is running dd-wrt.
> I'm pinging between that linux client and another linux client within my
> lan.
> 
> wireguard is running between the same linux client outside my lan and the
> same other linux client within my lan.  This
> time router is simply forwarding packets via NAT.
> 
> Openvpn ping times are much lower (about 10ms) and much lower variance than
> wireguard.  Wireguard pings
> are all over the place.
> 
> Packets coming in from the WAN are traversing some firewall that I don't
> control, which may be affecting results.  Openvpn
> is config to use udp.
>
> Any ideas?

Did you try using the same port for WG, as you use for OpenVPN (of course
stopping OpenVPN for a while to try that)?

It is possible that ISPs employ different priorities for particular UDP port
ranges;

Or they might do load-balancing across multiple links by src/dst/port/protocol,
and your WG's combination of that randomly happens to go via some unlucky
overloaded one.

-- 
With respect,
Roman
___
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: Endpoint address dns resolution - option to prefer IPv6 or IPv4

[Roman Mamedov]

On Sat, 14 Mar 2020 15:51:51 +0100
Torsten Krah  wrote:

> resend to the list:
> 
> Hm, sorry I don't get the message. Imho its down to the user. I can
> choose to use ping or ping6 or tell e.g. java via a system property to
> prefer IPv4 if dual stack is available.
> 
> In wireguard I can force ipv6 only by writing an ipv6 address in the
> endpoint, but via dns ... how to choose which one I prefer?

If it is so important to you to force one or the other, then make separate DNS
records for IPv4 and IPv6, server4.example.com, server6.example.com.

-- 
With respect,
Roman

Re: [ANNOUNCE] wireguard-linux-compat v1.0.20200330 released

[Roman Mamedov]

On Mon, 30 Mar 2020 18:19:17 -0600
"Jason A. Donenfeld"  wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
> 
> Hello,
> 
> A new version, v1.0.20200330, of the backported WireGuard kernel module for
> 3.10 <= Linux <= 5.5.y has been tagged in the git repository.

My kernel build for 5.4.29 has failed just now:

In file included from :
././net/wireguard/compat/compat.h:1029:20: error: redefinition of 
‘skb_reset_redirect’
 static inline void skb_reset_redirect(struct sk_buff *skb)
^~
In file included from ././net/wireguard/compat/compat.h:878,
 from :
./include/linux/skbuff.h:4538:20: note: previous definition of 
‘skb_reset_redirect’ was here
 static inline void skb_reset_redirect(struct sk_buff *skb)
^~
In file included from :
././net/wireguard/compat/compat.h: In function ‘skb_reset_redirect’:
././net/wireguard/compat/compat.h:1032:2: error: implicit declaration of 
function ‘skb_reset_tc’; did you mean ‘skb_reserve’? 
[-Werror=implicit-function-declaration]
  skb_reset_tc(skb);
  ^~~~
  skb_reserve
cc1: some warnings being treated as errors
scripts/Makefile.build:265: recipe for target 'net/wireguard/main.o' failed
make[3]: *** [net/wireguard/main.o] Error 1
scripts/Makefile.build:500: recipe for target 'net/wireguard' failed
make[2]: *** [net/wireguard] Error 2

> 
> == Changes ==
> 
>   * queueing: backport skb_reset_redirect change from 5.6
>   * version: bump
>   
>   This release has only one slight change, to put it closer to he 5.6 
> codebase,
>   but its main purpose is to bump us to a 1.0.y version number. Now that
>   WireGuard 1.0.0 has been released for Linux 5.6 [1], we can put the same 
> number on
>   the backport compat codebase.
>   
>   [1] https://lists.zx2c4.com/pipermail/wireguard/2020-March/005206.html
> 
> This release contains commits from: Jason A. Donenfeld.
> 
> As always, the source is available at 
> https://git.zx2c4.com/wireguard-linux-compat/
> and information about the project is available at https://www.wireguard.com/ .
> 
> This version is available in compressed tarball form here:
>   
> https://git.zx2c4.com/wireguard-linux-compat/snapshot/wireguard-linux-compat-1.0.20200330.tar.xz
>   SHA2-256: 2d57b239605be2ee0e4c2da935ff1a23e9ed8bb3ee692e10ae032ae50f280bef
> 
> A PGP signature of that file decompressed is available here:
>   
> https://git.zx2c4.com/wireguard-linux-compat/snapshot/wireguard-linux-compat-1.0.20200330.tar.asc
>   Signing key: AB9942E6D4A4CFC3412620A749FC7012A5DE03AE
>   Remember to unxz the tarball before verifying the signature.
> 
> If you're a package maintainer, please bump your package version. If you're a
> user, the WireGuard team welcomes any and all feedback on this latest version.
> 
> Finally, WireGuard development thrives on donations. By popular demand, we
> have a webpage for this: https://www.wireguard.com/donations/
> 
> Thank you,
> Jason Donenfeld
> 
> 
> -BEGIN PGP SIGNATURE-
> 
> iQJEBAEBCAAuFiEEq5lC5tSkz8NBJiCnSfxwEqXeA64FAl6CjGwQHGphc29uQHp4
> MmM0LmNvbQAKCRBJ/HASpd4DrqfwD/wJlEbOhYd1ixM3OI8Q2endXmJBRh9UimjL
> F2moHwTzDM49o3xQpfgQuBFbZWK0L/JNTSlKxrmBLcX9fBJ2VERT1Nrnlh414Ovw
> FLpmJt9gOWMF6hjlptXKaE/T0vRjzfLli2YzfzyvqMQg9hR/eRKlYhYWOu/fsm3L
> UtmBm8wGdDeo7e119M0dnfcfboW2b3NKQX87/bWrAn21BL9F+JsNx8Ytx2a5cU8z
> ZLj56sDWWclmoIXiLI1e+bKO9pXRXvfkSd11p5KK6knD8i8BtAn7uVVfda5VWxmO
> xooFhNq75photRM0t/VAKhp7ji96pYwSQD6Kw91HPgBcptB29XoacUcpLs40TUx5
> D7LpvYISKItZpPdfgSwIx3kyajBDmn8bpFZH8T+/cDsIvuJbdpGjP88Qr86JiKQ+
> BiVmTW5nXWn0d4tQIbw2w34BVre5cLheyWZN3Nk6f7bfxjba52Qa85rrjPoCNWv+
> PPsVTffIfAxk5ZavSuPUx+QMtmIqnC/bOb2WUn/+lukv+HVYYXO5GyHrfTqS9EW/
> kw/2dC9pGSye6bz7KYTQwRHSJnaA+SDk2ZNFdlrsYaoFfuZJeaFMkiU3xpIYGllv
> +v6rmQw/l0d4yZtHR6jmMV4qWeZetH0/5De21/syOQt8XuySjlkNVLRUPgIdZtKN
> ak69693rPw==
> =K43H
> -END PGP SIGNATURE-


-- 
With respect,
Roman

Re: Is there a way to use wireguard as a non-encrypted VPN?

[Roman Mamedov]

On Tue, 14 Apr 2020 17:02:41 +0200
ajs124  wrote:

> On Sat, 11 Apr 2020 12:13:36 -0700
>  wrote:
> 
> > I have some older routers that run OpenWRT just fine, but are a bit slow at
> > Wireguard (3-5 MBytes/s for SMB transfers) and which are too slow for
> > playing HD movies.
> > For these routers/uses I don't care about security, I just want a VPN to
> > tunnel (thru Comcast, and other ISPs that block lots of ports.)
> > If there was a way to use Wiireguard with encryption disabled, I'm pretty
> > sure my performance would be closer to 20-50 MB/s which would be more than
> > adequate.
> > Thanks.
> > Mike Farmwald
> > 
> 
> If you're actually just looking for an unencrypted tunnel, there is some 
> standardized stuff like GRE[1] or IP in IP[2] out there.
> 
> The Linux Kernel supports both of those natively and it looks to me like 
> OpenWRT should be able to configure at least one of them through its 
> interface.
> 
> 1: https://en.wikipedia.org/wiki/Generic_Routing_Encapsulation
> 2: https://en.wikipedia.org/wiki/IP_in_IP

Those both require dedicated IP on both ends of the connection, which is not
always the case on residential ISPs' IPv4 now.

I'd suggest to check out L2TP instead, which doesn't, and can be used without
encryption too, that one can work.

Or PPTP as mentioned, but it's more complex (separate signaling and data
protocols) for no good reason and has more issues traversing NATs/firewalls.

-- 
With respect,
Roman

Re: Significant Dropped Packets on WG interface

[Roman Mamedov]

On Thu, 14 May 2020 16:35:30 +0930
Mike O'Connor  wrote:

> Hi All
> 
> For the last few weeks my Wireguard link which I use to as my default
> gateway has been having issues with TCP connections stalling.
> 
> I've been trying to work out what is wrong. I just noticed that the
> Wireguard link has dropped packets at both ends.
> 
> wg-p2p    Link encap:UNSPEC  HWaddr
> 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
>   inet addr:104.127.123.10  P-t-P:103.127.123.10 
> Mask:255.255.255.248
>   inet6 addr: 2506:c500:ff4:1::ab/64 Scope:Global
>   inet6 addr: fe80::e6/64 Scope:Link
>   UP POINTOPOINT RUNNING NOARP  MTU:1500  Metric:1

Reduce MTU of the WG interfaces to accomodate for overhead. See 
https://www.mail-archive.com/wireguard@lists.zx2c4.com/msg01856.html for
calculations of by how much.

>   inet6 addr: 2506:c500:ff4:1::aa/64 Scope:Global

I wonder what's this IP range, is this some VPN service? Squatting on
unassigned IPs within 2000::/3 seems like a very bad practice. If they wanted
an imaginary GUA for their NAT66, I'd suggest something like 66::/16 instead.

-- 
With respect,
Roman

Re: [FR] How can I expose the wireguard tunnel as a socks5 proxy on the client?

[Roman Mamedov]

On Fri, 9 Oct 2020 17:00:31 +0330
Rudi C  wrote:

> > On Fri, Oct 9, 2020 at 4:52 PM Roman Mamedov  wrote:
> > just install a SOCKS proxy
> 
> These simple solutions get blocked by the DPI. (I do have my own VPS.)

Seems like you misunderstand what I mean. If you use the in-VPN (internal) IP
of your VPS, all communication with the SOCKS proxy installed on the VPS will
happen via the WireGuard tunnel. No DPI can look into that.

-- 
With respect,
Roman

Re: [FR] How can I expose the wireguard tunnel as a socks5 proxy on the client?

[Roman Mamedov]

On Fri, 9 Oct 2020 17:16:18 +0330
Rudi C  wrote:

> > On Fri, Oct 9, 2020 at 5:04 PM Roman Mamedov  wrote:
> > Seems like you misunderstand what I mean. If you use the in-VPN (internal) 
> > IP
> > of your VPS, all communication with the SOCKS proxy installed on the VPS 
> > will
> > happen via the WireGuard tunnel. No DPI can look into that.
> 
> You're right! Some questions:
> 1. What should I do client-side so that wireguard only covers my VPS's
> IP (and does not otherwise route traffic)? Will `AllowedIPs =
> SERVER_IP/32` do it?

SERVER_IP should be the in-VPN IP here, otherwise yes, and remove .0.0.0/0
and ::/0 from AllowedIPs.

> 2. How do I get the in-VPN IP of the server? Is it `Address` in `[Interface]`?

Yes. You can confirm via "ip addr list dev wgX" on the server.

> 3. I use ufw for the firewall on the server. Will ufw block my local
> machine? If not, with what IP should I set ufw rules? (My local
> machine doesn't have a static IP.) Of course, I could alternatively
> expose the socks proxy to the world with a password; How secure will
> that be?

Sorry, not familiar with ufw; generally you need to allow only connections
from the WG interface, or from the internal IP range (or just the "Address ="
of the client), and block all others.

-- 
With respect,
Roman

Re: [FR] How can I expose the wireguard tunnel as a socks5 proxy on the client?

[Roman Mamedov]

On Sun, 4 Oct 2020 15:41:52 +0330
Rudi C  wrote:

> I use Wireguard to circumvent Iran's censorship. A major problem with
> it is that it's very hard to selectively proxy specific domains/apps
> through Wireguard, while leaving others alone. This is an essential
> feature for Iran's internet, as:
> 1. The connection is terrible, so avoiding using the proxy for
> uncensored sites helps a lot.
> 2. International traffic is 2x more expensive, so avoiding the proxy
> for internal traffic is very beneficial.
> 3. Some internal sites ban international IPs and need Iranian IPs.
> 
> The easiest way to solve this program, as far as I understand, is to
> add the ability to expose the tunnel as a socks5 proxy on the client
> side. This is the approach that shadowsocks, v2ray, etc have adopted.
> There are mature solutions to selectively routing traffic through a
> socks proxy.
> 
> I searched around, and there are docker containers that already do
> this wireguard-to-socks thing; But running docker is expensive on a
> non-Linux machine, so it'd be much appreciated if you could support
> exposing socks and HTTP proxy servers natively.

If you tunnel to a VPS abroad, just install a SOCKS proxy on the remote end.
A good one is [1]. Then set the remote end's in-VPN IP and proxy port in your
apps to use.

[1] https://socks-relay.sourceforge.io/

To separate which sites use which proxy (or no proxy) SwitchSharp for Chrome
and FoxyProxy for Firefox, but you probably already know about those.

In case you meant connecting to commercial "VPN" services, then yes it
becomes a bit more complex, but you can try srelay on the local machine and
use the "-J" option, "outbound interface name". But I'm not sure if that would
just work on its own, or also needs some help from ip(6)tables or ip-rule.

-- 
With respect,
Roman

Re: [FR] How can I expose the wireguard tunnel as a socks5 proxy on the client?

[Roman Mamedov]

On Fri, 9 Oct 2020 16:19:22 +0200
Chris  wrote:

> Maybe I oversimplify your problem, but from what I read, your standard route 
> will be using the Iranian net.
> And - I guess - it is only a limited numer of IP addresses, that you would 
> like 
> to reach through the tunnel.
> 
> I don't know your OS, but simply adding ip routes pointing to the tunnel for 
> the 
> desired destinations would do the job.

OK, a desired destination would be *.youtube.com, how would you go about that?

You can't add routes to domain names of websites, not to mention to wildcards
of domain names; and websites can resolve into a lot of IPs, which will change
randomly due to load balancing, or due to sites migrating their hosting over
time. So just resolving them right now and using specific IPs likely wouldn't
work for long.

One solution is the browser extensions that I mentioned coupled with a SOCKS
proxy on remote side. Another is what David suggests with dnsmasq and ipset,
which seems like it'll be more transparent from the usage standpoint, but also
more complex to set up.

-- 
With respect,
Roman

Re: more specific routes for IPs added to "AllowedIPs =" ?

[Roman Mamedov]

On Wed, 30 Sep 2020 15:42:19 -0700
PGNet Dev  wrote:

> I've two linux machines connected with wg.
> 
>  Machine #1 is a remote VM, & connects to the public 'net.
> 
>  Machine #2 is local, on my LAN.
> 
> To date, they've only routed internal traffic. Nice -n- easy.
> 
> I'm adding forwarding of specific EXTERNAL traffic from the 'net, received at 
> Machine #1, to port-specific services, on the LAN.
> 
> E.g. a 'listener' on a local lan machine, @ ip 10.0.0.1 port 1
> 
> On the local end of the VPN, for any external IP that needs to traverse the 
> VPN link, adding the specific IP to
> 
>   AllowedIPs = ... X.X.X.X
> 
> works.  Traffic flows.
> 
> BUT, that adds a local route
>
>   X.X.X.X dev wg0 scope link
> 

"That" is called wg-quick. For any sort of non-basic usage, I suggest going
with "wg" directly. "wg" does not add any routes or IPs, it only sets up the
encrypted tunnel. The rest is completely up to you. Make your own wrapper
around it, that only does exactly what you need and nothing more.


> so ALL local traffic from local lan to that IP, e.g. an ssh session, gets 
> routed BACK via that new route over the VPN.
> 
> I'd like to limit that -- so that ONLY traffic from the 'net to that local 
> listener on ip 10.0.0.1 port 1 is routed back via the VPN; all _other_ 
> traffic to the originating IP (e.g., that ssh connection), gets routed over 
> my normal default route.
> 
> What's the cleanest way -- in wireguard config -- to
> 
>   (a) allow any/all IPs over the VPN
>   (b) limit the route to the specific ip target/port
> 
> So far, I seem to _need_ that originating IP in the "allowedips ="; which 
> creates the 'overreaching' route ...
> 
> I'm guessing some judicious use of PostUp/Down routes set?
> 
> 


-- 
With respect,
Roman

Re: Standardized IPv6 ULA from PublicKey

[Roman Mamedov]

On Mon, 29 Jun 2020 12:22:49 +0200
Toke Høiland-Jørgensen  wrote:

> Reid Rankin  writes:
> 
> > Each IPv6 network device is *required* to have a link-local
> > address by the RFC
> 
> Given this

What you quoted is the shakiest statement of the entire proposal. Might be a
cool idea and all, but I don't think RFCs say anything about "requiring" that
for point-to-point L3 interfaces, where there's no functioning multicast or
broadcast to begin with. And it doesn't seem nice that submitter is trying to
skew facts in their favor like that.

-- 
With respect,
Roman

Re: Standardized IPv6 ULA from PublicKey

[Roman Mamedov]

On Mon, 29 Jun 2020 13:03:40 +0200
Toke Høiland-Jørgensen  wrote:

> Eh? This is specified pretty clearly in RFC4291, section 2.1:

It also says:

-

2.5.6.  Link-Local IPv6 Unicast Addresses

   Link-Local addresses are for use on a single link.  Link-Local
   addresses have the following format:

   |   10 |
   |  bits| 54 bits |  64 bits   |
   +--+-++
   |111010|   0 |   interface ID |
   +--+-++


-

So should we also follow the designated format for link-locals, or accept that
WG's case differs from what they had in mind in those sections. That the
"interface" is a special one, with a "link" that doesn't function as other
kinds of links do, that there's no "neighbour" per se to contact by an
all-neighbour multicast for instance, no mechanism for the "all routers"
multicast to work, etc (i.e. all of what the LLs were intended to support).

To be clear I'm not against adding LLs, just that "the RFC says so" shouldn't
be considered the main argument for that when it comes to WG.

-- 
With respect,
Roman

Re: Wireguard on Ubuntu 18.04.4 (LTS)?

[Roman Mamedov]

On Mon, 20 Jul 2020 17:04:46 +0200
 wrote:

> Yes, it is up to date.
> Joachim
> 
> -Ursprüngliche Nachricht-
> Von: Jason A. Donenfeld  
> Gesendet: Monday, 20 July 2020 16:49
> An: Joachim Lindenberg 
> Cc: WireGuard mailing list 
> Betreff: Re: Wireguard on Ubuntu 18.04.4 (LTS)?
> 
> Is your system fully up to date? `apt update && apt upgrade`?

Seems like it's available only starting from Ubuntu 19.10:
https://packages.ubuntu.com/search?keywords=wireguard=names=all=all

-- 
With respect,
Roman

No longer compiles on 5.4.76

[Roman Mamedov]

Hello,

Building kernel 5.4.76 with WireGuard v1.0.20200908 fails for me now with:

  AS [M]  net/wireguard/crypto/zinc/chacha20/chacha20-x86_64.o
In file included from :
././net/wireguard/compat/compat-asm.h:44: warning: "SYM_FUNC_START" redefined
 #define SYM_FUNC_START ENTRY
 
In file included from ././net/wireguard/compat/compat-asm.h:9,
 from :
./include/linux/linkage.h:218: note: this is the location of the previous 
definition
 #define SYM_FUNC_START(name)\
 
In file included from :
././net/wireguard/compat/compat-asm.h:45: warning: "SYM_FUNC_END" redefined
 #define SYM_FUNC_END ENDPROC
 
In file included from ././net/wireguard/compat/compat-asm.h:9,
 from :
./include/linux/linkage.h:265: note: this is the location of the previous 
definition
 #define SYM_FUNC_END(name)\
 
net/wireguard/crypto/zinc/chacha20/chacha20-x86_64.S: Assembler messages:
net/wireguard/crypto/zinc/chacha20/chacha20-x86_64.S:123: Error: invalid 
character '(' in mnemonic
net/wireguard/crypto/zinc/chacha20/chacha20-x86_64.S:185: Error: invalid 
character '(' in mnemonic
net/wireguard/crypto/zinc/chacha20/chacha20-x86_64.S:187: Error: invalid 
character '(' in mnemonic
net/wireguard/crypto/zinc/chacha20/chacha20-x86_64.S:319: Error: invalid 
character '(' in mnemonic
net/wireguard/crypto/zinc/chacha20/chacha20-x86_64.S:1016: Error: invalid 
character '(' in mnemonic
net/wireguard/crypto/zinc/chacha20/chacha20-x86_64.S:1616: Error: invalid 
character '(' in mnemonic
net/wireguard/crypto/zinc/chacha20/chacha20-x86_64.S:1620: Error: invalid 
character '(' in mnemonic
net/wireguard/crypto/zinc/chacha20/chacha20-x86_64.S:1810: Error: invalid 
character '(' in mnemonic
net/wireguard/crypto/zinc/chacha20/chacha20-x86_64.S:1812: Error: invalid 
character '(' in mnemonic
net/wireguard/crypto/zinc/chacha20/chacha20-x86_64.S:1959: Error: invalid 
character '(' in mnemonic
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/gp10b.o
scripts/Makefile.build:348: recipe for target 
'net/wireguard/crypto/zinc/chacha20/chacha20-x86_64.o' failed
make[3]: *** [net/wireguard/crypto/zinc/chacha20/chacha20-x86_64.o] Error 1
scripts/Makefile.build:500: recipe for target 'net/wireguard' failed
make[2]: *** [net/wireguard] Error 2


-- 
With respect,
Roman

Re: No longer compiles on 5.4.76

[Roman Mamedov]

On Tue, 10 Nov 2020 18:56:56 +0500
Roman Mamedov  wrote:

> Hello,
> 
> Building kernel 5.4.76 with WireGuard v1.0.20200908 fails for me now with:
> 
>   AS [M]  net/wireguard/crypto/zinc/chacha20/chacha20-x86_64.o
> In file included from :
> ././net/wireguard/compat/compat-asm.h:44: warning: "SYM_FUNC_START" redefined
>  #define SYM_FUNC_START ENTRY
>  
> In file included from ././net/wireguard/compat/compat-asm.h:9,
>  from :
> ./include/linux/linkage.h:218: note: this is the location of the previous 
> definition
>  #define SYM_FUNC_START(name)\
>  
> In file included from :
> ././net/wireguard/compat/compat-asm.h:45: warning: "SYM_FUNC_END" redefined
>  #define SYM_FUNC_END ENDPROC
>  
> In file included from ././net/wireguard/compat/compat-asm.h:9,
>  from :
> ./include/linux/linkage.h:265: note: this is the location of the previous 
> definition
>  #define SYM_FUNC_END(name)\
>  
> net/wireguard/crypto/zinc/chacha20/chacha20-x86_64.S: Assembler messages:
> net/wireguard/crypto/zinc/chacha20/chacha20-x86_64.S:123: Error: invalid 
> character '(' in mnemonic
> net/wireguard/crypto/zinc/chacha20/chacha20-x86_64.S:185: Error: invalid 
> character '(' in mnemonic
> net/wireguard/crypto/zinc/chacha20/chacha20-x86_64.S:187: Error: invalid 
> character '(' in mnemonic
> net/wireguard/crypto/zinc/chacha20/chacha20-x86_64.S:319: Error: invalid 
> character '(' in mnemonic
> net/wireguard/crypto/zinc/chacha20/chacha20-x86_64.S:1016: Error: invalid 
> character '(' in mnemonic
> net/wireguard/crypto/zinc/chacha20/chacha20-x86_64.S:1616: Error: invalid 
> character '(' in mnemonic
> net/wireguard/crypto/zinc/chacha20/chacha20-x86_64.S:1620: Error: invalid 
> character '(' in mnemonic
> net/wireguard/crypto/zinc/chacha20/chacha20-x86_64.S:1810: Error: invalid 
> character '(' in mnemonic
> net/wireguard/crypto/zinc/chacha20/chacha20-x86_64.S:1812: Error: invalid 
> character '(' in mnemonic
> net/wireguard/crypto/zinc/chacha20/chacha20-x86_64.S:1959: Error: invalid 
> character '(' in mnemonic
>   CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/gp10b.o
> scripts/Makefile.build:348: recipe for target 
> 'net/wireguard/crypto/zinc/chacha20/chacha20-x86_64.o' failed
> make[3]: *** [net/wireguard/crypto/zinc/chacha20/chacha20-x86_64.o] Error 1
> scripts/Makefile.build:500: recipe for target 'net/wireguard' failed
> make[2]: *** [net/wireguard] Error 2

Seems related to:

commit 840d8c9b3e5f51d1005256e6c63eab4f81cbebfb
Author: Jiri Slaby 
Date:   Fri Oct 11 13:50:41 2019 +0200

linkage: Introduce new macros for assembler symbols

commit ffedeeb780dc554eff3d3b16e6a462a26a41d7ec upstream.

Introduce new C macros for annotations of functions and data in
assembly. There is a long-standing mess in macros like ENTRY, END,
ENDPROC and similar. They are used in different manners and sometimes
incorrectly.

So introduce macros with clear use to annotate assembly as follows:

a) Support macros for the ones below
   SYM_T_FUNC -- type used by assembler to mark functions
   SYM_T_OBJECT -- type used by assembler to mark data
   SYM_T_NONE -- type used by assembler to mark entries of unknown type

   They are defined as STT_FUNC, STT_OBJECT, and STT_NOTYPE
   respectively. According to the gas manual, this is the most portable
   way. I am not sure about other assemblers, so this can be switched
   back to %function and %object if this turns into a problem.
   Architectures can also override them by something like ", @function"
   if they need.

   SYM_A_ALIGN, SYM_A_NONE -- align the symbol?
   SYM_L_GLOBAL, SYM_L_WEAK, SYM_L_LOCAL -- linkage of symbols

b) Mostly internal annotations, used by the ones below
   SYM_ENTRY -- use only if you have to (for non-paired symbols)
   SYM_START -- use only if you have to (for paired symbols)
   SYM_END -- use only if you have to (for paired symbols)

c) Annotations for code
   SYM_INNER_LABEL_ALIGN -- only for labels in the middle of code
   SYM_INNER_LABEL -- only for labels in the middle of code

   SYM_FUNC_START_LOCAL_ALIAS -- use where there are two local names for
one function
   SYM_FUNC_START_ALIAS -- use where there are two global names for one
function
   SYM_FUNC_END_ALIAS -- the end of LOCAL_ALIASed or ALIASed function

   SYM_FUNC_START -- use for global functions
   SYM_FUNC_START_NOALIGN -- use for global functions, w/o alignment
   SYM_FUNC_START_LOCAL -- use for local functions
   SYM_FUNC_START_LOCAL_NOALIGN -- use for local functions, w/o
alignment
   SYM_FUNC_START_WEAK -- use for weak functions
   SYM_FUNC_START_WEAK_NOALIGN -- use fo

Re: OSX and Happy Eyeballs

[Roman Mamedov]

On Tue, 17 Nov 2020 13:00:01 +0100
"Marco Davids (SIDN)"  wrote:

> Hello,
> 
> We have a Wireguard VPN and everything is working fine.
> 
> There is just one little thing: IPv6 Happy Eyeballs.
> 
> Without the VPN enabled, happy eyeballs works fine. The  (IPv6) is 
> preferred over A (IPv4). But as soon as we enable the tunnel, it's the 
> other way around.
> 
> IPv6-only sites are perfectly reachable, but dual-stack sites are always 
> reached over IPv4.
> 
> It is not a showstopper, but I am just trying to understand why this is.
> 
> Anyone with the same experience and more knowledge about the inner 
> workings of Wireguard and Apple's happy eyeballs implementation that 
> would care to comment?

Do you use ULA IPs (fc00::/7) for the tunnel endpoints? Those are always
depreferred compared to IPv4. See RFC 6724:

https://tools.ietf.org/html/rfc6724#section-2.1

-- 
With respect,
Roman

Re: Should we sunset Windows 7 support?

[Roman Mamedov]

On Thu, 12 Nov 2020 09:34:43 +0100
"Jason A. Donenfeld"  wrote:

> Could you let me know the rationale for your continued use of Windows
> 7? Is it economic? Is it just UI preference, and security isn't a
> priority to you? Something else?

For me, the UI preference absolutely; but security *is* certainly a priority,
which is exactly why I will not run Windows 10: 
https://www.networkworld.com/article/2956574/windows-10-privacy-spyware-settings-user-agreement.html
https://www.forbes.com/sites/gordonkelly/2015/11/02/microsoft-confirms-unstoppable-windows-10-tracking/
https://blog.emsisoft.com/en/18770/the-truth-about-windows-10-spying-on-almost-everything-you-do/

Don't want to wrestle with disabling all of that, to find out that the next
update rolled back my settings, and then added more spying that can't be
disabled this time.

-- 
With respect,
Roman

Re: WG default routing

[Roman Mamedov]

On Tue, 5 Jan 2021 21:12:12 +0100
Chris Osicki  wrote:

> As far as I can see after few tests, AllowedIPs config file option has 
> nothing to do with routing and I hope 
> it will stay like this.

wg-quick uses AllowedIPs to also set up matching entries in the system routing
table. This can be disabled in its config.

> It is just a filter

It is not only a filter on incoming packets, but also WG's internal routing
table for knowing which packets should be sent to which peer.

-- 
With respect,
Roman

Re: mtu on Linux vs MacOS

[Roman Mamedov]

On Sun, 17 Jan 2021 11:36:42 +0100
Harald Dunkel  wrote:

> Hi folks,
> 
> I am using PPPoE to connect to my IP provider. To use wireguard on Linux I
> have to reduce the MTU in wg0.conf to 1400. Using the default 1420 a ssh
> connection tunneled through wireguard gets stuck (reproducible). An echo
> of a very long line (e.g. 4096 chars) is sufficient.
> 
> The weird part is, MTU 1420 works fine for wireguard on MacOS. How comes
> that Wireguard appears to be more stable on MacOS than on Linux?

WireGuard works just fine for me on Linux using MTU 1432 if over IPv4, and
1412 if over IPv6. The underlying PPPoE device MTU is 1492 in my case. Check
which PPPoE MTU you use in Linux vs in MacOS.

-- 
With respect,
Roman

Re: Multiple Clients behind NAT

[Roman Mamedov]

On Wed, 13 Jan 2021 20:14:46 +
"Posegga, Joachim"  wrote:

> Dear all,
> 
> I am trying to connect multiple wireguard clients behind the same NAT-Gateway 
> to a Mikrotik  server with a public IP. I am not yet sure where exactly the 
> problem is, but it seems that only one client at a time can establish a 
> tunnel. 
> 
> Is this a known problem due to the UDP transport, or should multiple clients 
> behind a NAT work in principle? 
> 
> I understand from the documentation that the server looks at the public key 
> of the incoming packet and identifies the client. The response sent back from 
> the server then arrives at the NAT gateway, and it should map the target port 
> to the correct client and forward it. However, I am not very familiar with 
> UDP over NAT, so I am wondering if this usually works without problems. If 
> this is the case, I would know that the problem is most likely on the side of 
> the Mikrotik server.

The NAT router will rewrite outgoing UDP port of your clients' packets when
they try to connect to other peers; for two clients on the LAN-side trying to
send from the same port, it should change that to two separate UDP ports.
Therefore remote peers will see your two clients as being on the same global
IP, but using two different ports -- and that should work normally.

Check with tcpdump on your NAT's WAN interface what actual UDP packets it
sends out. The ports also might be very different from what you specified in
WG's config, so account for that in firewalling or routing rules on remote
sides.

-- 
With respect,
Roman

Re: mtu on Linux vs MacOS

[Roman Mamedov]

On Thu, 21 Jan 2021 19:07:18 +0500
Roman Mamedov  wrote:

> On Sun, 17 Jan 2021 11:36:42 +0100
> Harald Dunkel  wrote:
> 
> > Hi folks,
> > 
> > I am using PPPoE to connect to my IP provider. To use wireguard on Linux I
> > have to reduce the MTU in wg0.conf to 1400. Using the default 1420 a ssh
> > connection tunneled through wireguard gets stuck (reproducible). An echo
> > of a very long line (e.g. 4096 chars) is sufficient.
> > 
> > The weird part is, MTU 1420 works fine for wireguard on MacOS. How comes
> > that Wireguard appears to be more stable on MacOS than on Linux?
> 
> WireGuard works just fine for me on Linux using MTU 1432 if over IPv4, and
> 1412 if over IPv6. The underlying PPPoE device MTU is 1492 in my case. Check
> which PPPoE MTU you use in Linux vs in MacOS.
> 

Oh, and I just remembered -- you're better off setting the same MTU on the
entire WG network, i.e. substract those 8 bytes of PPPoE interface on the
remote host's WG interface as well. Else the local interface MTU will need a
weird and so far unexplained further 4-byte reduction for things to work:
https://lists.zx2c4.com/pipermail/wireguard/2019-April/004078.html

-- 
With respect,
Roman

Re: Access subnet behind server.

[Roman Mamedov]

On Sat, 23 Jan 2021 11:52:56 -0500
Ken D'Ambrosio  wrote:

> Hey, all.  I'm relatively new to WireGuard, and have a RasPi at my house 
> doing firewall duty.  Installed WG on it, and on a VPS, and am trying to 
> get the VPS to access hosts on my home subnet.  So:
> 
> VPS <-192.168.50.0/24-> RasPi <--> [192.168.10.0/24]
> 
> And, clearly, I'm doing something wrong.
> 
> ---
> RasPi server/firewall:
> [Interface]
> Address = 192.168.50.1/24
> SaveConfig = false
> ListenPort = 51820
> PrivateKey = XXX
> [Peer]
> PublicKey = XXX
> AllowedIPs = 192.168.50.11/32
> 
> VPS:
> [Interface]
> Address = 192.168.50.11/24
> PrivateKey = XXX
> [Peer]
> PublicKey = XXX
> Endpoint = vpn.foo.bar:51820
> AllowedIPs = 192.168.50.0/24,192.168.10.0/24
> ---
> 
> The client connects just fine, and it can talk to the server's VPN IP 
> (192.168.50.1) as well as its internal interface (192.168.10.1).  
> Likewise, the server can talk to 192.168.50.11.  But nothing gets inside 
> to other 192.168.10.x hosts.  I do have forwarding set up for "all":
> 
> root@prouter:/proc# cat /proc/sys/net/ipv4/conf/all/forwarding
> 1
> 
> Note that the config files have gone through several permutations as I 
> tried to figure this out, so there may be some dumb stuff, but totally 
> open to suggestions right now.  I'm kinda stumped.  Note that a tcpdump 
> on the RasPi shows the ping requests coming in, but not being forwarded 
> to the internal interface, so I assume I'm just missing Something 
> Dumb(tm) in WG land.

Did you allow forwarding in RPi's firewall? Post "iptables-save" from it.


-- 
With respect,
Roman

Re: potentially disallowing IP fragmentation on wg packets, and handling routing loops better

[Roman Mamedov]

On Mon, 7 Jun 2021 11:34:21 +0200
"Jason A. Donenfeld"  wrote:

> 2) Local egress fragmentation WOULD be affected by this and is the
> most relevant thing in this discussion. In this case, a packet that
> gets encrypted and winds up being larger than the mtu of the interface
> that the encrypted packet will go out of gets fragmented. In this
> case, we could likely respond with an ICMP packet or similar in-path
> error. But keep in mind this whole situation is local: it usually will
> only happen out of misconfiguration. The best fix for the diagram I
> drew would be for the administrator to decrease the MTU of the
> wireguard interface to 1412.

In the L2 tunneling scenario the large VXLAN packets are generated locally, as
it will be common for the same host (aka "the router") to be both a WG peer
and a VXLAN VTEP, so it is going to be affected.

> So, of those concerned about this, which concerns are actually about
> (2) and (3)? Of those, which ones are about (2)? If you have concerns
> specifically about (2) that couldn't be fixed with reasonable system
> administration, I'd like to hear why and what the setup is that leads
> to that situation.

My described case is being able to transparently bridge two Ethernet LANs.

Hopefully the answer isn't "you don't really need to do that" or "apply
reasonable system administration and set up routing instead".

> As an aside, Roman asked about TTL. When tunneling, the outer packet
> header always must take the new TTL of the route to the tunnel
> endpoint, and not do anything with the potentially much smaller inner
> TTL.

As far as I can see the inner TTL is not smaller than usual on WG tunnels (64).
You could inherit it to the outside of the tunnel, like GRE does:
https://serverfault.com/questions/827239/gre-tunnel-ttl-number
But of course that's leaking a tiny bit of information about the encrypted
tunnel, dunno how critical that would be.

-- 
With respect,
Roman

Re: potentially disallowing IP fragmentation on wg packets, and handling routing loops better

[Roman Mamedov]

On Mon, 7 Jun 2021 13:27:10 +0200
"Jason A. Donenfeld"  wrote:

> Can you walk me through your use case a bit more, so I can wrap my mind
> around the requirements?
> 
> ingress --plain--> wireguard --wireguard[plain]--> vxlan 
> --vxlan[wireguard[plain]]--> egress

Not sure I understand your scheme correctly. In any case, the path of a
packet would be...

On peer 1:

* plain Ethernet -> wrapped into VXLAN -> encrypted into WireGuard

On peer 2:

* decrypted from WireGuard -> unwrapped from VXLAN -> plain Ethernet

> So my question is, why can't you set wireguard's MTU to 80 bytes less
> than vxlan's MTU? What's preventing that or making it infeasible?

To transparently bridge two Ethernet LANs, a VXLAN interface needs to join an
L2 bridge. All interfaces that are members of a bridge must have the same MTU.

As such, br0 members on both sides:
  eth0 (MTU 1500)
  vx0 (MTU 1500)

VXLAN transports full L2 frames encapsulating them into UDP. To fit the
full 1500-byte packet and accounting for VXLAN and related IP overheads,
the resulting packet size is 1574 bytes.

So this same host that just generated the 1574-byte encapsulated VXLAN packet
with something it received via its eth0 port, now needs to send it further to
its WG peer(s). For this to succeed, the in-tunnel WG MTU needs to be 1574 or
more, not 1412 or 1420, as VXLAN itself can't be fragmented[1]; or even if it
could, that would mean a much worse overhead ratio than currently.

[1] https://datatracker.ietf.org/doc/html/rfc7348#section-4.3

-- 
With respect,
Roman

Re: potentially disallowing IP fragmentation on wg packets, and handling routing loops better

[Roman Mamedov]

On Mon, 7 Jun 2021 16:46:17 +0500
Roman Mamedov  wrote:

> On Mon, 7 Jun 2021 13:27:10 +0200
> "Jason A. Donenfeld"  wrote:
> 
> > Can you walk me through your use case a bit more, so I can wrap my mind
> > around the requirements?
> > 
> > ingress --plain--> wireguard --wireguard[plain]--> vxlan 
> > --vxlan[wireguard[plain]]--> egress
> 
> Not sure I understand your scheme correctly. In any case, the path of a
> packet would be...
> 
> On peer 1:
> 
> * plain Ethernet -> wrapped into VXLAN -> encrypted into WireGuard
> 
> On peer 2:
> 
> * decrypted from WireGuard -> unwrapped from VXLAN -> plain Ethernet
> 
> > So my question is, why can't you set wireguard's MTU to 80 bytes less
> > than vxlan's MTU? What's preventing that or making it infeasible?
> 
> To transparently bridge two Ethernet LANs, a VXLAN interface needs to join an
> L2 bridge. All interfaces that are members of a bridge must have the same MTU.
> 
> As such, br0 members on both sides:
>   eth0 (MTU 1500)
>   vx0 (MTU 1500)
> 
> VXLAN transports full L2 frames encapsulating them into UDP. To fit the
> full 1500-byte packet and accounting for VXLAN and related IP overheads,
> the resulting packet size is 1574 bytes.
> 
> So this same host that just generated the 1574-byte encapsulated VXLAN packet
> with something it received via its eth0 port, now needs to send it further to
> its WG peer(s). For this to succeed, the in-tunnel WG MTU needs to be 1574 or
> more, not 1412 or 1420, as VXLAN itself can't be fragmented[1]; or even if it
> could, that would mean a much worse overhead ratio than currently.
> 
> [1] https://datatracker.ietf.org/doc/html/rfc7348#section-4.3

In case you are not convinced by this case, would you consider at least
allowing fragmentation when WG's in-tunnel MTU is set to >=1500? Because this
is the user effectively saying "yes I know this is not gonna fit in one
packet, I want to rely on WG packets being fragmented", but without the need
for extra knobs.

-- 
With respect,
Roman

Re: secondary IP on wg0 fails

[Roman Mamedov]

On Sat, 8 May 2021 17:31:58 +0100
lejeczek  wrote:

> I'm experiencing a pretty weird wireguard, or perhaps 
> kernel/OS stack bits behavior.
> 
> I have three nodes which all can ping each other on wg0's 
> IPs but when I add a secondary IP:
> 
> -> $ ip addr add 10.0.0.226/24 dev wg0
> 
> it gets weird, namely, say when that sec IP is on
> A -> B ping returns; C ping waits, no errors, no return
> B -> both C & A pings return
> C -> neither A nor B ping returns
> 
> I'm on CentOS with 4.18.0-301.1.el8.x86_64.
> All three nodes are virtually identical kvm VMs.
> 
> any suggestions as to what is not working here or how to 
> troubleshoot are vey appreciated.
> many thanks, L.

Did you add the new IP to AllowedIPs of that node on all the other nodes?

Also remember that sets of AllowedIPs should be unique within the network,
i.e. can't have the same AllowedIPs or ranges listed for multiple nodes at the
same time. Setting it to the same /24 on all nodes will not work.

If still not clear, better post your complete config (without keys).

-- 
With respect,
Roman

Re: secondary IP on wg0 fails

[Roman Mamedov]

On Sat, 8 May 2021 19:49:06 +0100
lejeczek  wrote:

> > Also remember that sets of AllowedIPs should be unique within the network,
> > i.e. can't have the same AllowedIPs or ranges listed for multiple nodes at 
> > the
> > same time. Setting it to the same /24 on all nodes will not work.
> >
> > If still not clear, better post your complete config (without keys).
> >
> It's the same single subnet 10.0.0.0/24 and to reiterate - 
> wg0's "primary" IPs can all ping each other.
> All nodes have, respectively:
> eg. node-B
> [peer]
> ...
> AllowedIPs = 10.0.0.1/32, 10.0.0.226/32
> Endpoint = 10.1.1.223:51851
> 
> [peer]
> ...
> AllowedIPs = 10.0.0.3/32, 10.0.0.226/32
> Endpoint = 10.1.1.225:51853

See above for "Also remember...", you cannot have 10.0.0.226/32 added to
multiple peers as AllowedIPs at the same time.

-- 
With respect,
Roman

Re: lost connection on dynamic IP

[Roman Mamedov]

On Thu, 20 May 2021 00:28:08 +0200
Vicente Bergas  wrote:

> There is a public IP assigned to the router. The IP is dynamic, so, it
> can change from time to time, but, once assigned, it is exclusive to
> the router.
> There is no carrier-grade NAT.
> I've configured the router to forward the wireguard port to the
> server, so, it is like the server is directly connected to the
> Internet.
> I think the PersistentKeepalive on the server side is not required. Is it?

I believe it is. Consider the server public IP has changed. The server sends
no Keepalives. The client sends them to the server's old IP. The whole thing
broke.

> So, what do you mean is that wireguard does a single DNS resolution at
> the beginning and further DNS resolutions need to be done elsewere. Is
> that correct?

Yes.

-- 
With respect,
Roman

Re: lost connection on dynamic IP

[Roman Mamedov]

On Tue, 18 May 2021 13:22:31 +0200
Vicente Bergas  wrote:

> A server connected to the Internet through an ISP that provides a
> dynamic IP with NAT.

If it's NAT, then your server has no dedicated public IP? What do you update
to DNS, IP of the ISP's NAT pool (shared IP with many other customers)?

> I think the issue happens when the ISP on the server side shuts down
> the Internet connection for more than 1 hour! Then, it is restored
> with a new IP.
> inadyn detects the new IP and updates the DNS.
> At this point the Internet connection is operational again, but the
> client remains disconnected until rebooted.
>
> Is this scenario expected to work due to the "Built-in Roaming" ?

It might work, helped by PersistentKeepalive, and as long as the server and the
client don't change their IPs/ports *at the same time*. To protect against
that, or to improve resiliency in general (and assuming there's actually no NAT
at the server side after all), your client should resolve the DNS record for
the server periodically, and in case the IP changed, call "wg set [interface]
peer [key] endpoint [IP:port]".

-- 
With respect,
Roman

Re: lost connection on dynamic IP

[Roman Mamedov]

On Thu, 20 May 2021 11:15:30 +0500
Roman Mamedov  wrote:

> > So, what do you mean is that wireguard does a single DNS resolution at
> > the beginning and further DNS resolutions need to be done elsewere. Is
> > that correct?
> 
> Yes.

I also remembered a case where just PersistentKeepalive won't save you, and
periodic DNS resolution on clients becomes mandatory. It is when the server's
physical location gets a power cut. On new boot-up (and router power-on) it
gets a new IP from the ISP, and has no idea where all the clients are. The
communication is broken until clients recheck the DNS record and update the
server's endpoint from that. WG does not do this on its own.

-- 
With respect,
Roman

Re: potentially disallowing IP fragmentation on wg packets, and handling routing loops better

[Roman Mamedov]

On Sun, 6 Jun 2021 11:13:36 +0200
"Jason A. Donenfeld"  wrote:

> Specifically the change would be to not allow IP fragmentation of the
> encrypted UDP packets. This way, in the case of a loop, eventually the
> packet size exceeds MTU, and it gets dropped: dumb and effective.
> Depending on how this discussion goes, a compromise would be to not
> allow fragmentation, but only for forwarded and kernel-generated
> packets, not not for locally generated userspace packets. That's more
> complex and I don't like it as much as just disallowing IP
> fragmentation all together.
> 
> Pros:
> - It solves the routing loop problem very simply.

Doesn't TTL already solve this?

> - Maybe people are running
> wireguard-over-gre-over-vxlan-over-l2tp-over-pppoe-over-god-knows-what-else,
> and this reduces the MTU to below 1280, yet they still want to put
> IPv6 through wireguard, and are willing to accept the performance
> implications.

Not only that. Sometimes transparent bridging of 1500 MTU LANs is required.

VXLAN does not allow tunnel endpoints to produce fragmented VXLAN packets.

With WG we can fragment them one level lower, *and* gain a higher efficiency
compared to hypothetical VXLAN's fragmentation, due to less header overhead on
2nd and further packets in a chain.

It would be unfortunate if this will become no longer possible.

It appears to me that people who might need to transparently join multiple
Ethernet LANs due to legacy network topologies they have to work with, weird
requirements, various legacy software etc, would outnumber those who even run
WG over WG at all, let alone getting themselves into a routing loop that way.

-- 
With respect,
Roman

Re: Multiple Keys per Peer

[Roman Mamedov]

On Sun, 02 May 2021 13:02:28 +0200
Nico Schottelius  wrote:

> when running a lot of VPN connections using wireguard, there are some
> questions we see quite often from users, two of which I'd like to
> discuss here:
> 
> Multiple keys per Peer
> --
> 
> Users often ask for sharing their connection with multiple
> devices. The obvious solution is for users to setup their own VPN
> endpoint with the first key and then reshare themselves. However, this
> is not feasible in many end user situations.

The prime and the most straightforward solution is to give each user multiple
keys, and let them connect from each endpoint as an independent Peer.

The rest of what you propose appears to be a set of bizarre hacks because
you don't want to do the above, because "(reasons)". Maybe start with
detailing those reasons first, or reconsidering if they are *really* that
important and unsurmountable :)

> Conceptually I see it problematic to assign multiple keys per Peer as
> the routing from outside ("where should this packet go to"?) might
> become ambiguous. One counter option would be to allow a peer to signal
> that it uses a certain part of the AllowedIPs. In comparison to layer 2
> networks, I see two approaches: 1) a bit similar to ARP/NDP, client
> addresses are learned 2) similar to dhcp-pd, clients "requesting" (in
> this context more: announcing) that they use a certain sub-range.
> 
> Protocol wise I'd imagine this to be rather simple:
> 
> side a: I want to use 2001:db8:a:b::/64
> side b:
>  - checking your allowed IPs covers that prefix -> no ignore
>  - checking whether the amount of sub routes is not exceeded
>  - and/or checking whether the sub-prefix length is of minimum size
>  (especially import for IPv6)
>  - yes: adjust routing table, insert more specific route
>  (with/without confirm probably should be modeld in tamarin)
> 
> What are your thoughts about an extension of wireguard with this?
> 
> If there are other suggestions to allow users to decide themselves how
> to split a range, let's say a /48 IPv6 network, without setting up their
> own redistribution node, I'd also be interested in hearing that.

-- 
With respect,
Roman

Re: wgX iface as slave to a bridge - Linux

[Roman Mamedov]

On Sat, 24 Apr 2021 11:11:50 +0100
lejeczek  wrote:

> Hi guys.
> 
> Apologies, I'll bother you guys as I failed to find some 
> better places to ask, I searched for forums etc. but failed.
> 
> Can wiregurard ifaces be enslaved by LInux bridge? I tried 
> but it did not work for me. Similarly "mavclan" - 
> would/should wireguard work that way?
> What I've tried and failed was on CentOS stream with 
> 4.18.0-294.el8.x86_64.

As others have replied, it is an L3 interface, not L2 which can join bridges.
One solution that many use is to run an L2 tunnel over WireGuard, such as
VXLAN or GRETAP. But then you lose even more MTU compared to the standard 1500.

-- 
With respect,
Roman

Re: NAT to NAT peers - 'EndPoint' IP data sharing among peers of the same key?

[Roman Mamedov]

On Sat, 3 Apr 2021 06:27:40 +0200
Giovanni Francesco  wrote:

> Hi, I am looking to understand if "EndPoint" IP data may be shared among 
> peers within the tunnel?
> 
> The question may sound confusing, let me explain my setup.
> 
> I have a static IPv4 wireguard server (let's call it "A" peer) which has two 
> downstream WG clients peers "B" and "C" on remote networks with dynamic WAN 
> IPs (roaming).
> In my current configuration all my clients "B" and "C" have a single peer "A" 
> - therefore all traffic must always go to "A" - "A" is in a datacenter in 
> another country.
> 
> "B" and "C" have dynamic every changing IP "EndPoint" information, in my 
> current setup this is not a problem because "A" is a static host.
> 
> If "B" and "C" are connected to "A" - is it possible for me to make B and C 
> peers of eachother without "EndPoint" ?
> In other words, if B public key is a peer of C and vise versa would its 
> connection to "A" share the IP addresses ("EndPoint" or where to go) 
> downstream to "B" and "C" so they can establish direct connectivity or would 
> traffic always need to continue to traverse via "A"?

No, peer A will not tell peer B the current IP/port of peer C.

Check out other tools, for instance Tinc can do this, but not WG.

-- 
With respect,
Roman

Re: T-Mobile 4G/5G CGNAT vs WireGuard tunnel jitter

[Roman Mamedov]

On Sat, 10 Apr 2021 10:27:23 -0500
Lonnie Abelbeck  wrote:

> I have been testing the T-Mobile Home Internet (4G/5G fixed wireless) service 
> to a Linode VM via WireGuard.
> 
> The TMHI service uses CGNAT plus an additional NAT in their modem/gateway 
> with a MTU of 1420, so WireGuard is configured with a 1340 MTU.

Do they provide IPv6? I see mentions that yes, but with incoming connections
blocked. Might still work for WG.

-- 
With respect,
Roman

Re: ipv6 connexion fail - ipv4 OK

[Roman Mamedov]

On Thu, 26 Aug 2021 13:14:00 +0200
Daniel  wrote:

> Correction
> 
> Le 25/08/2021 à 17:25, Daniel a écrit :
> > Hi list,
> > 
> > I setup wireguard on a server running Debian 11 and get it to work with 
> > 2 clients (Debian 11 and Ubuntu 20.04). Clients and server are on 
> > separate networks, one client behind a FW the other direct on Internet, 
> > no FW at all (VPS).
> > 
> > With this setup and ipv4 connection to the public IP of the server, 
> > everything is working as expected, ipv4 as well as ipv6 are passing 
> > smoothly.
> > 
> > Now I want to connect using the ipv6 address of the wg interface as both 
> > clients and server have ULA ipv6.
> 
> Here is GUA to read.
> 
> > This fail, wg show that connection is 
> > established but VPN is not usable. It's not a FW problem as I can ssh to 
> > the ipv6 address, as well as a netcat test from/to server IP -from each 
> > client- on an UDP port is working properly. Also, 
> > net.ipv6.conf.all.forwarding=1 is activated in sysctl.conf
> > 
> > All network stuff is done in /etc/network/interfaces which call the 
> > config file. The ipv6 address of the server is affected _to the 
> > wireguard interface_ (in ipv4 it's another interface who take care of 
> > the public address)
> > 
> > Server version is wireguard-tools v1.0.20210223.
> > 
> > If someone have any hint, thanks to share ;)
> 

IPv6 requires the in-WG MTU to be 20 bytes less than when running over IPv4.
Try reducing it accordingly.

-- 
With respect,
Roman

Re: [Warning: DMARC Fail Email] Re: ipv6 connexion fail - ipv4 OK

[Roman Mamedov]

On Sat, 28 Aug 2021 07:05:45 +0930
Mike O'Connor  wrote:

> On a 1500 link I'm having to use 1280 to get ipv6 to successfully go 
> over a wireguard link.

Then it is not a true 1500 MTU link, something in-between drops packets at a
lower bar. Or maybe not all of them, but just UDP, for example.

But yeah, 1280 is worth trying as well, maybe Daniel has a similar issue.

As for me I am using MTU 1412 WG over IPv6 on a 1492 MTU underlying link just
fine.

> I really think wireguard should be able to fragment and send via 
> multiply UDP packets.

It is able to, as long as the underlying link MTU is set to a correct value
(i.e. where largest-sized packets actually go through, and not get silently
dropped).

-- 
With respect,
Roman

Re: enabling WG0 allows telegram but impedes browsing

[Roman Mamedov]

On Fri, 20 Aug 2021 13:16:34 +0200
S Bauer  wrote:

> Hello team,
> 
> Hoping you could help me out with a foggy situation.
> The past week I have been struggling to get the Wireguard VPN working
> smoothly. Everything seems to work on paper, except in a specific way
> it doesn't. I am using Pop!_OS 21.04 (Ubuntu Hirsute).
> 
> SitRep;
> I work as a freelance consultant and want to be careful about the
> local networks' peeping tom when accessing sensitive work documents
> from 'out of office', e.g. at a friend's place or at a hotel. So my
> objective is to access my home network via PiHole and then continue
> onward to access my work-related documents on a fileserver.
> I was hoping this could be easily achieved with Wireguard.
> 
> Using the Wireguard VPN wg0 with wg-quick worked perfectly when I
> connected to my brother's phone hotspot (4G). I could access our home
> via VPN as expected and could work on my documents without any
> problems.
> The trouble is that I am now at a different location, working with a
> fixed router from Ziggo NL. For some reason the WG0 still connects
> perfectly, but after that a small mystery occurs. I did not make any
> modifications to WG0.conf, so I remain stumped.
> With WG active, I am no longer able to access any webpage. So no
> access to protonmail\gmail, reddit or anything else. Telegram,
> however, is still working fine. Internal machines on the home's local
> network (IP-camera) can also be accessed directly.
> Disabling the WG gives me full access to any webpage as usual. So
> something is amiss that affects my browser only (Firefox 91.0).

What's your MTU on the wg0 interface? Try reducing that to 1400, or if
that doesn't help, to 1280.

-- 
With respect,
Roman

Re: Wireguard Neighborhood (IPv6)

[Roman Mamedov]

On Fri, 24 Sep 2021 11:31:40 -0400
tlhackque  wrote:

> WireGuard server (Linux, details below) behind a site router that
> handles IPv4 NAT & an IPv6 tunnel.
> 
> Server LAN has other hosts (and multiple subnets/vlans) - mostly dual stack.
> 
> The WireGuard server is able to access the WireGuard peers (clients)
> over IPv6.  The other hosts (and the router) are not.
> 
> The clients can't even ping the other hosts - the echo replies are
> generated, but they end up with an icmp6 unreachable.
> 
> It turns out that the other hosts (and router) send an icmp6 Neighbor
> Solicitation for the clients, which is never answered.
> 
> My interim solution was to implement
> https://github.com/setaou/ndp-proxy, which will respond with Neighbor
> Advertisements for the entire WireGuard subnet.
> 
> This is a rather crude solution - since ndp-proxy doesn't know what
> clients are connected, and since it requires one proxy process/wg interface.
> 
> It seems to me that WireGuard (in this case on the server) should at
> least be responding to Neighbor Solicitations for AllowedIPs of its
> active peers... Of course in the case of a WireGuard tunnel between two
> such sites, this is symmetric.
> 
> I did look at net.ipv6.conf.*.proxy_ndp, but that requires adding each
> address - and in any case I couldn't get it to work.  Neither did
> advertising the server as a "router" with radvd.
> 
> Unless I'm missing something, it seems to me that supporting NDP is the
> simplest "it just works" approach in any case...

You are not configuring your network correctly routing-wise, and the issue is
not "WireGuard not supporting NDP" -- yes it doesn't, but that's not the point
to blame for the behavior that you observe -- which is completely normal.

Server LAN is one L2 network, the WG network is *another* and L3 network.
There is nothing nowhere that dictates that there would be NDP replies
*across* separate networks, let alone L2 vs L3.

The WG network needs its own separate IPv6 range, and other hosts need to have
a route to that range "via" the VPN server (if its not their default gateway).
Then, the WG clients need to know the route back to those other hosts, i.e.
the network they use needs to be in AllowedIPs for the VPN server.

-- 
With respect,
Roman

Re: linux: bridging/bonding not possible

[Roman Mamedov]

On Thu, 14 Oct 2021 04:45:32 +0200
uxdwzco...@moenia.de wrote:

> as I understand, linux needs the ability to change hardware-addresses on
> netdevs to put them into a bridge or bond, but wireguard-netdevs on
> linux don't support hw-addresses at all (at least in kernel 5.10).
> 
> is it possible (or even planned) to add hw-addresses to the
> wireguard-netdevs or does this interfere with the concept of wireguard?

Hello,

It is not a matter of hw-addresses;

Wireguard is L3 interface, transferring IPv4 and IPv6 packets.

For bridging you would need an L2 interface, which transfers Ethernet frames.

It is possible to do a bridge with WG, by using an L2-over-L3 tunnel such as
VXLAN or GRETAP over WG, and bridging that. Of course this leads to additional
overhead and MTU reduction.

If you would prefer to have an L2 VPN directly, there are other solutions such
as Tinc and OpenVPN.

-- 
With respect,
Roman

Re: WireGuard with obfuscation support

[Roman Mamedov]

On Mon, 27 Sep 2021 02:11:30 -0500
Bruno Wolff III  wrote:

> On Mon, Sep 27, 2021 at 09:53:08 +0900,
>   Nico Schottelius  wrote:
> >
> >I'd appreciate if wireguard upstream would take this in, maybe even
> >supporting multiple / dynamic listen ports.
> 
> The problem is mostly orthogonal to Wireguard. There isn't going to be a 
> one size fits all solution for hiding traffic. Failures in hiding traffic 
> are potentially very bad for individuals. As such general solutions are 
> not something you can recommend universally to people, as amateurs are 
> not going to be able to make good decisions about the risks and some may 
> get themselves tortured and killed.
> 
> This may not be something the developers for Wireguard want to be 
> responsible for.

No need to make DPI's job especially easy either.

Don't over-estimate the resources available to DPIs, if there aren't easy
ways to block, it might be almost as good as unblockable.

And it is far from all cases that hiding traffic would result in bad
consequences. Just hiding it enough so it evades the dumb automated filter,
many people will thank you.

As far as I understand right now WireGuard is very vulnerable to being
blocked, due to the fixed 4 bytes at the beginning(?) of each packet. At least
that needs to be randomized/encrypted. Just make the entire packet look like
random noise - that's the sign of good crypto, right? It is even somewhat
surprising as to why that's not the case already.

-- 
With respect,
Roman

Re: WireGuard with obfuscation support

[Roman Mamedov]

On Mon, 27 Sep 2021 04:14:35 -0500
Bruno Wolff III  wrote:

> This isn't a simple problem. The assumption is that someone is seeing 
> your network traffic and blocking it.

The assumption is that there's an appliance at the ISP which has a DROP rule
for UDP with 4 fixed bytes at a fixed offset. It has five hundreds other rules
to process as well, so it can't spend "too much" time on specifically WG.

> They are still going to see it  even if you disguise it.

With obfuscation there would be UDP packets of random junk, and it would be a
much harder job to come up with a rule to drop those without affecting
anything else.

> So you are going to need to disquise it as  something that whoever is
> watching isn't going to care about. That is going to vary a lot depending on
> who is watching. You may also need to hide who you are communicating with.
> In some cases that will be even more important.

You are going full-on "Enemy of the state" movie. The reality is most often a
lot simpler and more benign.

> There are going to be a number of ways to detect Wireguard traffic and 
> it is pretty unlikely that the bar for detection can be raised enough to 
> be relevant with a few simple changes to the protocol.

That's not a justification for not trying at all.

-- 
With respect,
Roman

Re: [Warning: DMARC Fail Email] Re: ipv6 connexion fail - ipv4 OK

[Roman Mamedov]

On Mon, 30 Aug 2021 19:28:11 +0200
Daniel  wrote:

> To be sure (and I think it is as I have no problem with ipv4):
> 
> . my interfaces are named wig4tootai our wigserver Nothing wrong here ?
> 
> . conf file are not named .conf but server.conf or 
> anyname.conf Nothing wrong here too ?

Interface names are arbitrary. As for proper .conf filenames on your system, I
am not sure, but you can simply check whether everything looks fine by running
the "wg" tool.

> Conserning the setup, I made another one using one VPS (one public ipv4 and
> one ipv6 /64 range) but get the same result. No FW involved at all :(

Do you get WG working at all, between some other two hosts (not involving this
particular server for now)?

-- 
With respect,
Roman

Re: [Warning: DMARC Fail Email] Re: ipv6 connexion fail - ipv4 OK

[Roman Mamedov]

On Mon, 30 Aug 2021 19:44:21 +0200
Daniel  wrote:

> > Do you get WG working at all, between some other two hosts (not involving 
> > this
> > particular server for now)?
> Yes. Clients are shown on both sides as connected, trafic seems to go 
> out on each side but other one as received near to nothing.

I mean not just "shown as connected", but have you got actual traffic working
between any two hosts. Even just forgetting this server for a while. So that
you can rule out some general issue and concentrate on just the particular
machine setup.

-- 
With respect,
Roman

Re: [Warning: DMARC Fail Email] Re: ipv6 connexion fail - ipv4 OK

[Roman Mamedov]

On Mon, 30 Aug 2021 12:24:01 +0200
Daniel  wrote:

> Using tcpdump -i any I see the trafic coming to the gre interface and 
> that's all. But netstat show
> 
> udp6   0  0 :::12345 :::*    
> 0  125391 -
> 
> and ps aux output is
> 
> dh@peech:~$ ps ax|grep wg
>     6969 ?    I< 0:00 [wg-crypt-wig4to]
>     7026 ?    I  0:00 [kworker/1:2-wg-kex-wig4tootai]
> 
> Question: is wireguard really listening on all ipv6 addresses ? If not, 
> how is the address choosen ?

Yes it does.


You seem to have some very complex setup, I suggest to look into whether you
send replies from the interface you expect them to. If you use wg-quick, maybe
switch to just wg and set up manually and with careful intent of each action,
as wg-quick might not have in mind some aspect of your setup.

-- 
With respect,
Roman

Re: WireGuard Windows should have default MTU of 1280.

[Roman Mamedov]

On Tue, 22 Feb 2022 00:57:10 +0500
Roman Mamedov  wrote:

> On Mon, 21 Feb 2022 22:16:22 +0300
> Michael Tokarev  wrote:
> 
> > 21.02.2022 22:11, Michael Adams wrote:
> > > Throwing in my two cents: I was using MTU 1280 on Tinc a few years back, 
> > > for IPv6 VPN support on Windows & Linux. It's good practice.
> > 
> > Lemme guess. The OP is routing wg packets over IPv6?  Can this be
> > the problem here, because V6 has larger overhead so that 1420 is
> > too large to fit into 1500 bytes together with IPv6 header?
> 
> 1420 is picked specifically so that it fits into a 1500 byte packet with IPv6.
> 
> If you run WG exclusively over IPv4, you can use up to 1432.

Correction: 1440.

https://www.mail-archive.com/wireguard@lists.zx2c4.com/msg01856.html

I'm just used to subtracting 8 everywhere, because my ISP *does* use PPPoE. :)

-- 
With respect,
Roman

Re: WireGuard Windows should have default MTU of 1280.

[Roman Mamedov]

On Mon, 21 Feb 2022 22:16:22 +0300
Michael Tokarev  wrote:

> 21.02.2022 22:11, Michael Adams wrote:
> > Throwing in my two cents: I was using MTU 1280 on Tinc a few years back, 
> > for IPv6 VPN support on Windows & Linux. It's good practice.
> 
> Lemme guess. The OP is routing wg packets over IPv6?  Can this be
> the problem here, because V6 has larger overhead so that 1420 is
> too large to fit into 1500 bytes together with IPv6 header?

1420 is picked specifically so that it fits into a 1500 byte packet with IPv6.

If you run WG exclusively over IPv4, you can use up to 1432.

However, if your ISP uses, say, PPPoE or L2TP, you need to reduce these
numbers accordingly, as the underlying interface will not have the full 1500
byte MTU.

-- 
With respect,
Roman

Re: [WireGuard] Header / MTU sizes for Wireguard

[Roman Mamedov]

On Thu, 24 Aug 2023 08:50:20 -0400
Saint Michael  wrote:

> This is the Achiles' heel of Wireguard. It reduces the MTU too much. Other
> tunneling techniques use a much larger MTU. I use Mikotik routers and one
> of the supported tunnels goes up to 1472. Some apps requiere a large MTU.
> Why Wireguard requieres so much space, so to speak?

Because it uses encryption, and each packet is also cryptographically signed.

I believe the other tunnels you have in mind will transfer data in plaintext
(unencrypted).

-- 
With respect,
Roman

Re: [WireGuard] Header / MTU sizes for Wireguard

[Roman Mamedov]

On Thu, 17 Aug 2023 20:14:52 +
blurt_overkill...@simplelogin.com wrote:

> I see here[1] that if you're using IPv4 exclusively, you can get away with
> an MTU of 1440. If my client only has IPv4 internet, however the server
> issues an IPv6 address for use by the client, can the client still use 1440
> without fragmentation, or must the client use 1420, because even though
> their connection is IPv4, they are issued an IPv6 address within the tunnel?
> 
> [1] https://lists.zx2c4.com/pipermail/wireguard/2017-December/002201.html

Yes they can. This is only affected by whether or not WG itself runs over
v4/v6, not whether you use v4 or v6 inside WG.

Be aware though that some residential Internet connections use MTU-reducing
tunnels for ISP authentication. The most popular one would be PPPoE with 8
bytes that you need to substract, but there also can be L2TP or PPTP with
larger overheads.

-- 
With respect,
Roman

Re: [PATCH] wg-quick: linux: add restart command.

[Roman Mamedov]

On Wed, 16 Aug 2023 07:06:53 +0200
Henrik Hautakoski  wrote:

> Add a simple "restart" command that just do cmd_down followed by an cmd_up. 
> Saves abit of typing :)
> 
> Signed-off-by: Henrik Hautakoski 
> ---
>  src/wg-quick/linux.bash | 7 ++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/src/wg-quick/linux.bash b/src/wg-quick/linux.bash
> index 69e5bef..cc9f288 100755
> --- a/src/wg-quick/linux.bash
> +++ b/src/wg-quick/linux.bash
> @@ -298,7 +298,7 @@ execute_hooks() {
>  
>  cmd_usage() {
>   cat >&2 <<-_EOF
> - Usage: $PROGRAM [ up | down | save | strip ] [ CONFIG_FILE | INTERFACE ]
> + Usage: $PROGRAM [ up | down | restart | save | strip ] [ CONFIG_FILE | 
> INTERFACE ]
>  
> CONFIG_FILE is a configuration file, whose filename is the interface 
> name
> followed by \`.conf'. Otherwise, INTERFACE is an interface name, with
> @@ -373,6 +373,11 @@ elif [[ $# -eq 2 && $1 == down ]]; then
>   auto_su
>   parse_options "$2"
>   cmd_down
> +elif [[ $# -eq 2 && $1 == restart ]]; then
> + auto_su
> + parse_options "$2"
> + cmd_down
> +cmd_up

cmd_down and the lines prior use a TAB to indent, but cmd_up uses 4 spaces
instead.

-- 
With respect,
Roman

Re: How to improve Wireguard speed?

[Roman Mamedov]

On Wed, 1 Jun 2022 10:07:31 +0100
Houman  wrote:

> I didn't change the MTU settings, but I have a suspicion about MTU. I
> found this article here that makes some interesting suggestions to set
> MTU to 1280: https://keremerkan.net/posts/wireguard-mtu-fixes/
> 
> And beyond that iptables -A FORWARD -p tcp --tcp-flags SYN,RST SYN -j
> TCPMSS --clamp-mss-to-pmtu

So did you apply both of that, and what was the effect?

What are the other point that you test against, is it another VPS (better if
you could try with that), or your home connection?

It could be your home provider has different speed limits (shaping) in place
for UDP. Should be possible to test this with:

  iperf3 -s# on VPS

  iperf3 -u -b 500M -c  -R # on the other side

And then see how many "Lost/Total Datagrams" (xx %) you get. A high percentage
would indicate that the actual top speed for UDP is less than 500Mbit by this
value.

-- 
With respect,
Roman

Re: Outgoing ping required in container environment (even with PersistentKeepalive)

[Roman Mamedov]

On Sun, 08 May 2022 08:34:46 +0200
Nico Schottelius  wrote:

> The connection stays correctly established.
> 
> If anyone has a pointer on what might be going on, any help is
> appreciated.

Maybe you don't have a corresponding firewall rule, and happen to rely on the
ESTABLISHED,RELATED matching instead? This could explain the "works after a
ping" behaviour. But then the PersistentKeepalive could be expected to help as
well.

-- 
With respect,
Roman

Re: [Question or feature request] Support multiple peer config file using something like /etc/wireguard/conf.d

[Roman Mamedov]

Hello,

On Tue, 19 Jul 2022 21:36:57 +
Quentin Vallin  wrote:

> I'm trying to separate my peer configuration and automate it. 
> 
> I know that I can use the post hook PostUp = wg addconf /path/to/my/file
> 
> It would be easier to have a special path were wireguard can merge the config 
> file together, like /etc/wireguard/conf.d//.conf. 

Personally I use my own shell script that concatenates pieces of the config
into a single file and runs wg on that. The same script also then handles
addresses, routes and such.

If you're doing any sort of non-trivial setup, you'd likely have a similar
wrapper on top of WG, and then it is easy to also make your own "conf.d".

-- 
With respect,
Roman

Re: [RESEND PATCH v3] wg: Support restricting address family of DNS resolved Endpoint

[Roman Mamedov]

On Sun, 19 Feb 2023 19:04:28 +0100
Daniel Gröber  wrote:

> +static inline bool parse_address_family(int *family, const char *value)
> +{
> + if (strcmp(value, "inet") == 0)
> + *family = AF_INET;
> + else if (strcmp(value, "inet6") == 0)
> + *family = AF_INET6;

Wouldn't the first condition match "inet6" as well, not ever checking the
second condition?

> + else if (strcmp(value, "unspec") == 0)
> + *family = AF_UNSPEC;
> + else
> + return false;
> +
> + return true;
> +}


-- 
With respect,
Roman

Re: Source IP incorrect on multi homed systems

[Roman Mamedov]

On Sun, 19 Feb 2023 21:18:34 +0100
Nico Schottelius  wrote:

> If I am not mistaken that would mean in practice:
> 
>if orignal_pkg.ip_dst == one_of_my_ips then
>   return_pkg.ip.src = orignal_pkg.ip_dst
>   return_pkg.ip.dst = orignal_pkg.ip_src
>fi
> 
> For me that sounds like a sane approach (aside from
> my very simplified algorithm).

Except there is no request and response in WG, and as such no original or
return packet. Another peer contacts you, then some time later you contact the
other peer. Or the other way round.

WG-wise what will need to be done is to store in the each peer's information
structure the local IP that we are supposed to use for communication with that
peer; and updating it when receiving packets from the peer, using the
destination of those. So you would see a "Local IP" in each "peer" section
when doing a "wg show".

Also, until there is such IP initially stored, it will have to be some default
outgoing IP of the system towards that peer. BTW, how would this work in your
setup, what if not the peer contacts you first, but your machine needs to
contact the peer?

-- 
With respect,
Roman

Force a specific IP for outgoing WG traffic with SNAT?

[Roman Mamedov]

Hello,

I'm trying to move all my WG communication with peers to a non-primary IP of my 
server.

It has IPs added like this:

inet6 2001:db8::ca6c/128 scope global deprecated 
   valid_lft forever preferred_lft 0sec
inet6 2001:db8::1/128 scope global nodad 
   valid_lft forever preferred_lft forever

What I tried:

  ip6tables -t nat -I POSTROUTING -d 2000::/3 -p udp --dport 51820 -j SNAT 
--to-source 2001:db8::ca6c

Also tried to filter by --sport, and also briefly without a port filter at all.

This has zero effect, as shown by tcpdump all the WG traffic still originates 
from 2001:db8::1

Does anyone have an idea why is that? Thanks

-- 
With respect,
Roman