Re: [PATCH] Update default QoS markers for ssh
Just for the record the latest OpenBSD snapshot with the changed ssh / sshd
causes ssh connections to fail
across a VMWare Fusion 10 NAT network.
ssh into the box you get a banner sent before the connection is closed.
ssh out of the like updating a CVS tree gets you...
Obtaining OpenBSD-current src tree.
packet_write_wait: Connection to 130.102.79.219 port 22: Broken pipe
cvs [update aborted]: end of file from server (consult above messages if any)
Finished obtaining OpenBSD-current src tree.
Adding
IPQoS lowdelay
to both the ssh_config / ssh_config make this work as before.
Machines on the Internal network can SSH into and out amongst each other but
going across the NAT boundary to the real world fails.
With the state of VMWare (the company) I doubt there will be a fix for Fusion
anytime soon.
Leaving this here for the archives.
Ian McWilliam
From: owner-t...@openbsd.org <owner-t...@openbsd.org> on behalf of Stuart
Henderson <s...@spacehopper.org>
Sent: Monday, 2 April 2018 12:06 AM
To: Job Snijders
Cc: tech@openbsd.org
Subject: Re: [PATCH] Update default QoS markers for ssh
On 2018/04/01 13:29, Job Snijders wrote:
> On Sun, Apr 01, 2018 at 11:29:55AM +0100, Stuart Henderson wrote:
> > On 2018/03/31 16:10, Job Snijders wrote:
> > > TL;DR: I propose to update the defaults to use DSCP "AF21" (Low
> > > Latency Data) for interactive session traffic, and CS1 ("Lower
> > > Effort") for non-interactive traffic. This applies to both IPv4 and
> > > IPv6.
> >
> > I think this is the right thing to do, but needs handling in
> > conjunction with changes to PF, which has dual queue-setting
> > (IPTOS_LOWDELAY packets go in one queue along with empty TCP ACKs,
> > other packets can go in another lower priority queue).
> >
> > Since firewalls are often updated less often than hosts I think it
> > would be better if the PF side was done first with some time to give
> > people chance to update, though it doesn't need to be too long (they
> > can set qos in sshd_config if they want to retain the old setting).
> >
> > A few other pieces use IPTOS_LOWDELAY (pfsync, carp, vxlan, etherip)
> > which I think would want switching too.
>
> What we're looking at is some kind of "migrate TOS to DSCP" project.
> There is a few tens of files where TOS values need to be examined and an
> appropiate DSCP value choosen.
Yes, that's clearly a much bigger scope. But if we're starting with ssh
those should be on the radar too IMO.
>I'm not sure I'd go as far as blindly
> replacing IPTOS_LOWDELAY with IPTOS_DSCP_AF21, but perhaps as a
> transition that is what is needed in some places.
Indeed. For some of these uses (like carp) something like CS7 seems
more appropriate. ssh seems a great place to start gaining more
experience with this - in particular any breakage can be handled
more easily than things like carp (since ssh already has config
options for this).
> We can start by treating IPTOS_LOWDELAY and IPTOS_DSCP_AF21 similarly in
> pf (see untested patch below), and then accept patches that migrate from
> TOS to DSCP?
That approach generally makes sense to me. It seems that some other
codepoints should probably also be treated as higher priority but
I'm not quite sure which yet...
> As far as I understand, out-of-the-box pf doesn't do anything with TOS
> values, so users wlil have to have explicitly configured something on
> the firewall or clients anyhow?
Yes, the firewall needs to have queues enabled and traffic assigned with
"set queue (foo, bar)" syntax.
> Kind regards,
>
> Job
>
> share/man/man5/pf.conf.5 | 12 +---
> sys/net/pf.c | 8 +---
> 2 files changed, 14 insertions(+), 6 deletions(-)
>
> diff --git share/man/man5/pf.conf.5 share/man/man5/pf.conf.5
> index 13e23423daa..f5a65e1b0d4 100644
> --- share/man/man5/pf.conf.5
> +++ share/man/man5/pf.conf.5
> @@ -679,7 +679,9 @@ interface, the queueing priority will also be written as
> the priority
> code point in the 802.1Q VLAN header.
> If two priorities are given, TCP ACKs with no data payload and packets
> which have a TOS of
> -.Cm lowdelay
> +.Cm lowdelay ,
> +or DSCP value
> +.Cm af21 ,
> will be assigned to the second one.
> Packets with a higher priority number are processed first,
> and packets with the same priority are processed
> @@ -702,7 +704,9 @@ section.
> Packets matching this rule will be assigned to the specified
> .Ar queue .
> If two queues are given, packets which have a TOS of
> -.Cm lowdelay
> +.Cm lowdelay ,
> +or DSCP value
> +.Cm af21 ,
> and TCP ACKs with no data payload will be assigned to the second one.
Re: [PATCH] Update default QoS markers for ssh
On 2018/04/01 08:59, Theo de Raadt wrote: > > I think this is the right thing to do, but needs handling in conjunction > > with changes to PF, which has dual queue-setting (IPTOS_LOWDELAY packets > > pf has to change first?? > > I don't understand the requirement that pf must be capable of handling > this naunce of packets, before any of our applications are changed. > > other applications outside openbsd have already been adapted to use > DSCP. > > there is traffic on the internet doing this, and yet noone has died. Why not? PF has a feature which is *exactly* designed to work with this, specifically so ssh still works when the line is flooded, and the minimum fix is a couple of lines of diff..
Re: [PATCH] Update default QoS markers for ssh
> I think this is the right thing to do, but needs handling in conjunction > with changes to PF, which has dual queue-setting (IPTOS_LOWDELAY packets pf has to change first?? I don't understand the requirement that pf must be capable of handling this naunce of packets, before any of our applications are changed. other applications outside openbsd have already been adapted to use DSCP. there is traffic on the internet doing this, and yet noone has died.
Re: [PATCH] Update default QoS markers for ssh
On 2018/04/01 13:29, Job Snijders wrote:
> On Sun, Apr 01, 2018 at 11:29:55AM +0100, Stuart Henderson wrote:
> > On 2018/03/31 16:10, Job Snijders wrote:
> > > TL;DR: I propose to update the defaults to use DSCP "AF21" (Low
> > > Latency Data) for interactive session traffic, and CS1 ("Lower
> > > Effort") for non-interactive traffic. This applies to both IPv4 and
> > > IPv6.
> >
> > I think this is the right thing to do, but needs handling in
> > conjunction with changes to PF, which has dual queue-setting
> > (IPTOS_LOWDELAY packets go in one queue along with empty TCP ACKs,
> > other packets can go in another lower priority queue).
> >
> > Since firewalls are often updated less often than hosts I think it
> > would be better if the PF side was done first with some time to give
> > people chance to update, though it doesn't need to be too long (they
> > can set qos in sshd_config if they want to retain the old setting).
> >
> > A few other pieces use IPTOS_LOWDELAY (pfsync, carp, vxlan, etherip)
> > which I think would want switching too.
>
> What we're looking at is some kind of "migrate TOS to DSCP" project.
> There is a few tens of files where TOS values need to be examined and an
> appropiate DSCP value choosen.
Yes, that's clearly a much bigger scope. But if we're starting with ssh
those should be on the radar too IMO.
>I'm not sure I'd go as far as blindly
> replacing IPTOS_LOWDELAY with IPTOS_DSCP_AF21, but perhaps as a
> transition that is what is needed in some places.
Indeed. For some of these uses (like carp) something like CS7 seems
more appropriate. ssh seems a great place to start gaining more
experience with this - in particular any breakage can be handled
more easily than things like carp (since ssh already has config
options for this).
> We can start by treating IPTOS_LOWDELAY and IPTOS_DSCP_AF21 similarly in
> pf (see untested patch below), and then accept patches that migrate from
> TOS to DSCP?
That approach generally makes sense to me. It seems that some other
codepoints should probably also be treated as higher priority but
I'm not quite sure which yet...
> As far as I understand, out-of-the-box pf doesn't do anything with TOS
> values, so users wlil have to have explicitly configured something on
> the firewall or clients anyhow?
Yes, the firewall needs to have queues enabled and traffic assigned with
"set queue (foo, bar)" syntax.
> Kind regards,
>
> Job
>
> share/man/man5/pf.conf.5 | 12 +---
> sys/net/pf.c | 8 +---
> 2 files changed, 14 insertions(+), 6 deletions(-)
>
> diff --git share/man/man5/pf.conf.5 share/man/man5/pf.conf.5
> index 13e23423daa..f5a65e1b0d4 100644
> --- share/man/man5/pf.conf.5
> +++ share/man/man5/pf.conf.5
> @@ -679,7 +679,9 @@ interface, the queueing priority will also be written as
> the priority
> code point in the 802.1Q VLAN header.
> If two priorities are given, TCP ACKs with no data payload and packets
> which have a TOS of
> -.Cm lowdelay
> +.Cm lowdelay ,
> +or DSCP value
> +.Cm af21 ,
> will be assigned to the second one.
> Packets with a higher priority number are processed first,
> and packets with the same priority are processed
> @@ -702,7 +704,9 @@ section.
> Packets matching this rule will be assigned to the specified
> .Ar queue .
> If two queues are given, packets which have a TOS of
> -.Cm lowdelay
> +.Cm lowdelay ,
> +or DSCP value
> +.Cm af21 ,
> and TCP ACKs with no data payload will be assigned to the second one.
> See
> .Sx QUEUEING
> @@ -1608,7 +1612,9 @@ Normally only one
> .Ar queue
> is specified; when a second one is specified it will instead be used for
> packets which have a TOS of
> -.Cm lowdelay
> +.Cm lowdelay ,
> +or DSCP value
> +.Cm af21 ,
> and for TCP ACKs with no data payload.
> .Pp
> To continue the previous example, the examples below would specify the
> diff --git sys/net/pf.c sys/net/pf.c
> index d841f834af1..aac20603a01 100644
> --- sys/net/pf.c
> +++ sys/net/pf.c
> @@ -2804,7 +2804,7 @@ pf_build_tcp(const struct pf_rule *r, sa_family_t af,
> h->ip_len = htons(tlen);
> h->ip_v = 4;
> h->ip_hl = sizeof(*h) >> 2;
> - h->ip_tos = IPTOS_LOWDELAY;
> + h->ip_tos = IPTOS_DSCP_AF21;
> h->ip_len = htons(len);
> h->ip_off = htons(ip_mtudisc ? IP_DF : 0);
> h->ip_ttl = ttl ? ttl : ip_defttl;
> @@ -7020,7 +7020,8 @@ done:
> pf_scrub(pd.m, s->state_flags, pd.af, s->min_ttl,
> s->set_tos);
> pf_tag_packet(pd.m, s->tag, s->rtableid[pd.didx]);
> - if (pqid || (pd.tos & IPTOS_LOWDELAY)) {
> + if (pqid || (pd.tos & IPTOS_LOWDELAY)
> + || (pd.tos & IPTOS_DSCP_AF21)) {
> qid = s->pqid;
> if (s->state_flags & PFSTATE_SETPRIO)
>
Re: [PATCH] Update default QoS markers for ssh
On Sun, Apr 01, 2018 at 11:29:55AM +0100, Stuart Henderson wrote:
> On 2018/03/31 16:10, Job Snijders wrote:
> > TL;DR: I propose to update the defaults to use DSCP "AF21" (Low
> > Latency Data) for interactive session traffic, and CS1 ("Lower
> > Effort") for non-interactive traffic. This applies to both IPv4 and
> > IPv6.
>
> I think this is the right thing to do, but needs handling in
> conjunction with changes to PF, which has dual queue-setting
> (IPTOS_LOWDELAY packets go in one queue along with empty TCP ACKs,
> other packets can go in another lower priority queue).
>
> Since firewalls are often updated less often than hosts I think it
> would be better if the PF side was done first with some time to give
> people chance to update, though it doesn't need to be too long (they
> can set qos in sshd_config if they want to retain the old setting).
>
> A few other pieces use IPTOS_LOWDELAY (pfsync, carp, vxlan, etherip)
> which I think would want switching too.
What we're looking at is some kind of "migrate TOS to DSCP" project.
There is a few tens of files where TOS values need to be examined and an
appropiate DSCP value choosen. I'm not sure I'd go as far as blindly
replacing IPTOS_LOWDELAY with IPTOS_DSCP_AF21, but perhaps as a
transition that is what is needed in some places.
We can start by treating IPTOS_LOWDELAY and IPTOS_DSCP_AF21 similarly in
pf (see untested patch below), and then accept patches that migrate from
TOS to DSCP?
As far as I understand, out-of-the-box pf doesn't do anything with TOS
values, so users wlil have to have explicitly configured something on
the firewall or clients anyhow?
Kind regards,
Job
share/man/man5/pf.conf.5 | 12 +---
sys/net/pf.c | 8 +---
2 files changed, 14 insertions(+), 6 deletions(-)
diff --git share/man/man5/pf.conf.5 share/man/man5/pf.conf.5
index 13e23423daa..f5a65e1b0d4 100644
--- share/man/man5/pf.conf.5
+++ share/man/man5/pf.conf.5
@@ -679,7 +679,9 @@ interface, the queueing priority will also be written as
the priority
code point in the 802.1Q VLAN header.
If two priorities are given, TCP ACKs with no data payload and packets
which have a TOS of
-.Cm lowdelay
+.Cm lowdelay ,
+or DSCP value
+.Cm af21 ,
will be assigned to the second one.
Packets with a higher priority number are processed first,
and packets with the same priority are processed
@@ -702,7 +704,9 @@ section.
Packets matching this rule will be assigned to the specified
.Ar queue .
If two queues are given, packets which have a TOS of
-.Cm lowdelay
+.Cm lowdelay ,
+or DSCP value
+.Cm af21 ,
and TCP ACKs with no data payload will be assigned to the second one.
See
.Sx QUEUEING
@@ -1608,7 +1612,9 @@ Normally only one
.Ar queue
is specified; when a second one is specified it will instead be used for
packets which have a TOS of
-.Cm lowdelay
+.Cm lowdelay ,
+or DSCP value
+.Cm af21 ,
and for TCP ACKs with no data payload.
.Pp
To continue the previous example, the examples below would specify the
diff --git sys/net/pf.c sys/net/pf.c
index d841f834af1..aac20603a01 100644
--- sys/net/pf.c
+++ sys/net/pf.c
@@ -2804,7 +2804,7 @@ pf_build_tcp(const struct pf_rule *r, sa_family_t af,
h->ip_len = htons(tlen);
h->ip_v = 4;
h->ip_hl = sizeof(*h) >> 2;
- h->ip_tos = IPTOS_LOWDELAY;
+ h->ip_tos = IPTOS_DSCP_AF21;
h->ip_len = htons(len);
h->ip_off = htons(ip_mtudisc ? IP_DF : 0);
h->ip_ttl = ttl ? ttl : ip_defttl;
@@ -7020,7 +7020,8 @@ done:
pf_scrub(pd.m, s->state_flags, pd.af, s->min_ttl,
s->set_tos);
pf_tag_packet(pd.m, s->tag, s->rtableid[pd.didx]);
- if (pqid || (pd.tos & IPTOS_LOWDELAY)) {
+ if (pqid || (pd.tos & IPTOS_LOWDELAY)
+ || (pd.tos & IPTOS_DSCP_AF21)) {
qid = s->pqid;
if (s->state_flags & PFSTATE_SETPRIO)
pd.m->m_pkthdr.pf.prio = s->set_prio[1];
@@ -7032,7 +7033,8 @@ done:
} else {
pf_scrub(pd.m, r->scrub_flags, pd.af, r->min_ttl,
r->set_tos);
- if (pqid || (pd.tos & IPTOS_LOWDELAY)) {
+ if (pqid || (pd.tos & IPTOS_LOWDELAY)
+ || (pd.tos & IPTOS_DSCP_AF21)) {
qid = r->pqid;
if (r->scrub_flags & PFSTATE_SETPRIO)
pd.m->m_pkthdr.pf.prio = r->set_prio[1];
Re: [PATCH] Update default QoS markers for ssh
On 2018/03/31 16:10, Job Snijders wrote:
> TL;DR: I propose to update the defaults to use DSCP "AF21" (Low Latency
> Data) for interactive session traffic, and CS1 ("Lower Effort") for
> non-interactive traffic. This applies to both IPv4 and IPv6.
I think this is the right thing to do, but needs handling in conjunction
with changes to PF, which has dual queue-setting (IPTOS_LOWDELAY packets
go in one queue along with empty TCP ACKs, other packets can go in another
lower priority queue).
Since firewalls are often updated less often than hosts I think it would
be better if the PF side was done first with some time to give people
chance to update, though it doesn't need to be too long (they can set
qos in sshd_config if they want to retain the old setting).
A few other pieces use IPTOS_LOWDELAY (pfsync, carp, vxlan, etherip)
which I think would want switching too.
> interactive sessions and the second for non-interactive sessions.
> The default is
> -.Cm lowdelay
> +.Cm af21
> +.Ar (Low-Latency Data)
> for interactive sessions and
> -.Cm throughput
> +.Cm CS1
> +.Ar (Lower Effort)
Mismatching caps/lower case (between these two and with the "accepted
values" list).
[PATCH] Update default QoS markers for ssh
Dear all,
There may be opportunity for improvement of ssh(1) and sshd(8)'s default
QoS markers for better integration in environments that can offer either
layer-2 or layer-3 prioritisation profiles. Currently ssh(1) and sshd(8)
set obsoleted values 'lowdelay' for interactive sessions and
'throughput' for non-interactive sessions.
TL;DR: I propose to update the defaults to use DSCP "AF21" (Low Latency
Data) for interactive session traffic, and CS1 ("Lower Effort") for
non-interactive traffic. This applies to both IPv4 and IPv6.
The openbsd 'lowdelay' value translates to IP TOS '0x10', and
'throughput' translates to IP TOS 0x08'. The IP TOS field is 8 bits, and
was defined in RFC 1349 (1992 A.D.). IP TOS was deprecated in RFC 2474
(1998 A.D.) with the introduction of "Differentiated Services Field" aka
DSCP.
When the world transitioned from TOS to DSCP, the IP TOS field was
chopped up into two parts: the 6 most significant bits became the
DSField (which contains a DSCP), and the 2 least significant bits became
a place for ECN experimentation. (Similarly in IPv6, the 'Traffic Class'
became the DSField). To convert the IP TOS value to a DSCP value you
shift the value 2 bits to the right: 'lowdelay' = DSCP 0x04 and
'throughput' = DSCP 0x02. It should be noted that DSCP 0x04 and DSCP
0x02 have no meaning. Both 0x02 and 0x04 map to Precedence "routine",
which is the /least/ important Precedence.
Using IP TOS values is problematic because network operators can't match
on TOS values on today's equipment. We looked at Cisco IOS XR, Juniper
Junos, and Nokia SR-OS - and there aren't classifiers to match on TOS
values. Instead, the systems I mentioned allow you to match on DSCP
value (and perhaps IPv4 precedence).
The ability to correctly match QoS markers of this nature is important
in controlled environments, you'll generally see that the DSCP values
are fully trusted, or not honored at all (such as when crossing Internet
boundaries). An example would be how traffic from VOIP phones/apps are
treated in enterprise office networks, perhaps even in the entire branch
network. The industry practise is that iff QoS markers are honored, they
are translated 1:1 - so AF21 goes into AF21 (or equivalent).
I selected AF21 as this is the highest priority within the low-latency
service class (and it is higher than what we have today). SSH is elastic
and time-sensitive data, where a user is waiting for a response via the
network in order to continue with a task at hand. As such, these flows
should be considered foreground traffic, with delays or drops to such
traffic directly impacting user-productivity.
You may ask, "why not EF?" - the risk with EF is that you can easily end
up in a tiny policer queue, as an example some CPEs by default only
allow up to 100Kbps of EF traffic. EF is meant for inelastic traffic,
where for instance the codec sends packets at the rate the codec
produces them, regardless of availability of capacity.
For bulk SSH traffic, the "Lower Effort" marker was chosen to enable
networks implementing a scavanger/lower-than-best effort class to
discriminate scp(1) below normal activities, such as web surfing. In
general this type of bulk SSH traffic is a background activity. Today
this traffic would go into 'best effort', potentially drowning out other
traffic on wireless links, so kicking this into 'background' may improve
the user experience.
An advantage of using "AF21" for interactive SSH and "CS1" for bulk SSH
is that they are recognisable values on all common platforms (IANA
https://www.iana.org/assignments/dscp-registry/dscp-registry.xml ), and
for AF21 specifically a rigorous definition of the intended behavior
exists https://tools.ietf.org/html/rfc4594#section-4.7 in addition to
the definition of the Assured Forwarding PHB group
https://tools.ietf.org/html/rfc2597, and for CS1 (Lower Effort) there is
https://tools.ietf.org/html/rfc3662 - finally this document is also a
recommened reading: https://tools.ietf.org/html/rfc8325
Another advantage is that the first three bits of "AF21" map to the
equivalent I 802.1D PCP, IEEE 802.11e, MPLS EXP/Cos and IP
Precedence value of 2 (also known as "Immediate", or "AC_BE"), and CS1's
first 3 bits map to I 802.1D PCP, IEEE 802.11e, MPLS/Cos and IP
Precedence value 1 ("Background" or "AC_BK").
The below patch ensures that in environments where people are using QoS,
the openssh defaults can easily matched both for interactive and bulk
SSH traffic in all layers of the transport stack (wired/wirless
Ethernet, MPLS, IP) and priortize accordingly.
Kind regards,
Job
The below patch should be portable, but I've only tested it on my own
openbsd machines.
usr.bin/ssh/readconf.c| 4 ++--
usr.bin/ssh/servconf.c| 4 ++--
usr.bin/ssh/ssh_config.5 | 6 --
usr.bin/ssh/sshd_config.5 | 6 --
4 files changed, 12 insertions(+), 8 deletions(-)
diff --git usr.bin/ssh/readconf.c usr.bin/ssh/readconf.c
index 5d17b725600..d3a121373d8 100644
---
