Re: unexpected behavior with pf queues (bandwidth allocations seemingly ignored)

[David Dahlberg]

Please try first to remove „min“. „Min“ makes it a „real-time service curve“ in 
HFSC terminology, which may react … „unexpectedly“ when exceeded. And you do 
not want „real-time“ properties for file transfer anyways.

> Am 24.07.2021 um 00:21 schrieb Scott Lewandowski :
> 
> I am attempting to prioritize traffic from a particular host. I have the 
> following queue definitions, with this match rule:
> 
> queue rootq on $ext_if bandwidth 13M max 13M
> queue file1_bak parent rootq bandwidth 10M min 8M qlimit 1024
> queue std parent rootq bandwidth 3M min 2M default qlimit 1024
> 
> match from 192.168.1.176 set queue file1_bak
> 
> However, even when the host at .176 has a steady stream of data to output, it 
> is not being prioritized for bandwidth utilization. For example:
> 
> fw0# pfctl -v -sq 
> queue rootq on vmx0 bandwidth 13M, max 13M
>  [ pkts:  0  bytes:  0  dropped pkts:  0 bytes:  0 ]
>  [ qlength:   0/ 50 ]
> queue file1_bak parent rootq bandwidth 10M, min 8M qlimit 1024
>  [ pkts:   1279  bytes:1825459  dropped pkts:  0 bytes:  0 ]
>  [ qlength:   0/1024 ]
> queue std parent rootq bandwidth 3M, min 2M default qlimit 1024
>  [ pkts:   8994  bytes:   12333179  dropped pkts:  0 bytes:  0 ]
>  [ qlength:   2/1024 ]
> 
> Even after an extended period of execution, I see similar results. The 
> supposedly prioritized host sees upload speeds of 17-200KB/s, whereas other 
> hosts see 800KB/s or more.
> 
> I do not understand the behavior I am seeing. Why are other hosts being 
> allocated so much bandwidth for uploads? 
> 
> Also of interest is that when I added the queues, a host that reliably used 
> to have consistent 27MB/s downloads now sees variable speeds between 13 and 
> 24MB/s, even when there is no other (meaningful) network activity. I'm not 
> sure if this is related to the upload speed issue I am seeing. I realize 
> there is outbound control traffic from the downloading host, but I can't 
> imagine that should be impacted by the queues when there is no other 
> meaningful network traffic. To try to address the download issue, I've 
> experimented with adding a control traffic queue and assigning traffic to 
> (std, ctrl), but that hasn't helped (in fact, it's hurt).
> 
> Based on some past threads I've read on related issues, I've tried adding 
> "max" specifications to each queue, but that hasn't helped, and it doesn't 
> seem it should be necessary based on the docs. Oddly, if I specify a max of 
> 13 on each rule -- with no suffix, which I accidentally did -- I seem to get 
> the desired behavior, but in that case pf obviously isn't enforcing the max 
> correctly, and I also see download speeds of less than 1KB/s. Adding the 
> intended suffix gives the same observable behavior as I saw without the max 
> specifier at all. 
> 
> I am running up-to-date 6.9 on ESX 6.7 with vmxnet3 vNICs. The VM has 2 vCPUs 
> and 1G and is showing no sign of resource constraints.
> 
> Any help or thoughts would be appreciated!
> 

Re: unexpected behavior with pf queues (bandwidth allocations seemingly ignored)

[Scott Lewandowski]

Hi David, thanks for your reply. I had initially tried the rule without a max 
or min specification, and was not seeing the desired behavior. I just 
reconfirmed that using the rules without a min specified exhibits the 
unexpected behavior:

queue rootq on $ext_if bandwidth 13M max 13M
queue file1_bak parent rootq bandwidth 10M qlimit 1024
queue std parent rootq bandwidth 3M default qlimit 1024

fw0# pfctl -v -sq 
queue rootq on vmx0 bandwidth 13M, max 13M
  [ pkts:  0  bytes:  0  dropped pkts:  0 bytes:  0 ]
  [ qlength:   0/ 50 ]
queue file1_bak parent rootq bandwidth 10M qlimit 1024
  [ pkts:   6719  bytes:9497288  dropped pkts:  0 bytes:  0 ]
  [ qlength:   0/1024 ]
queue std parent rootq bandwidth 3M default qlimit 1024
  [ pkts:  75265  bytes:  103153102  dropped pkts:  0 bytes:  0 ]
  [ qlength:   9/1024 ]


> -Original Message-
> From: David Dahlberg 
> Sent: Saturday, July 24, 2021 8:47 AM
> To: Scott Lewandowski 
> Cc: misc@openbsd.org
> Subject: Re: unexpected behavior with pf queues (bandwidth allocations
> seemingly ignored)
> 
> Please try first to remove „min“. „Min“ makes it a „real-time service curve“ 
> in
> HFSC terminology, which may react … „unexpectedly“ when exceeded. And
> you do not want „real-time“ properties for file transfer anyways.
> 
> > Am 24.07.2021 um 00:21 schrieb Scott Lewandowski
> :
> >
> > I am attempting to prioritize traffic from a particular host. I have the
> following queue definitions, with this match rule:
> >
> > queue rootq on $ext_if bandwidth 13M max 13M queue file1_bak parent
> > rootq bandwidth 10M min 8M qlimit 1024 queue std parent rootq
> > bandwidth 3M min 2M default qlimit 1024
> >
> > match from 192.168.1.176 set queue file1_bak
> >
> > However, even when the host at .176 has a steady stream of data to
> output, it is not being prioritized for bandwidth utilization. For example:
> >
> > fw0# pfctl -v -sq
> > queue rootq on vmx0 bandwidth 13M, max 13M
> >  [ pkts:  0  bytes:  0  dropped pkts:  0 bytes:  0 ]
> >  [ qlength:   0/ 50 ]
> > queue file1_bak parent rootq bandwidth 10M, min 8M qlimit 1024
> >  [ pkts:   1279  bytes:1825459  dropped pkts:  0 bytes:  0 ]
> >  [ qlength:   0/1024 ]
> > queue std parent rootq bandwidth 3M, min 2M default qlimit 1024
> >  [ pkts:   8994  bytes:   12333179  dropped pkts:  0 bytes:  0 ]
> >  [ qlength:   2/1024 ]
> >
> > Even after an extended period of execution, I see similar results. The
> supposedly prioritized host sees upload speeds of 17-200KB/s, whereas
> other hosts see 800KB/s or more.
> >
> > I do not understand the behavior I am seeing. Why are other hosts being
> allocated so much bandwidth for uploads?
> >
> > Also of interest is that when I added the queues, a host that reliably used
> to have consistent 27MB/s downloads now sees variable speeds between 13
> and 24MB/s, even when there is no other (meaningful) network activity. I'm
> not sure if this is related to the upload speed issue I am seeing. I realize 
> there
> is outbound control traffic from the downloading host, but I can't imagine
> that should be impacted by the queues when there is no other meaningful
> network traffic. To try to address the download issue, I've experimented
> with adding a control traffic queue and assigning traffic to (std, ctrl), but 
> that
> hasn't helped (in fact, it's hurt).
> >
> > Based on some past threads I've read on related issues, I've tried adding
> "max" specifications to each queue, but that hasn't helped, and it doesn't
> seem it should be necessary based on the docs. Oddly, if I specify a max of 13
> on each rule -- with no suffix, which I accidentally did -- I seem to get the
> desired behavior, but in that case pf obviously isn't enforcing the max
> correctly, and I also see download speeds of less than 1KB/s. Adding the
> intended suffix gives the same observable behavior as I saw without the max
> specifier at all.
> >
> > I am running up-to-date 6.9 on ESX 6.7 with vmxnet3 vNICs. The VM has 2
> vCPUs and 1G and is showing no sign of resource constraints.
> >
> > Any help or thoughts would be appreciated!
> >

unexpected behavior with pf queues (bandwidth allocations seemingly ignored)

[Scott Lewandowski]

I am attempting to prioritize traffic from a particular host. I have the 
following queue definitions, with this match rule:

queue rootq on $ext_if bandwidth 13M max 13M
queue file1_bak parent rootq bandwidth 10M min 8M qlimit 1024
queue std parent rootq bandwidth 3M min 2M default qlimit 1024

match from 192.168.1.176 set queue file1_bak

However, even when the host at .176 has a steady stream of data to output, it 
is not being prioritized for bandwidth utilization. For example:

fw0# pfctl -v -sq 
queue rootq on vmx0 bandwidth 13M, max 13M
  [ pkts:  0  bytes:  0  dropped pkts:  0 bytes:  0 ]
  [ qlength:   0/ 50 ]
queue file1_bak parent rootq bandwidth 10M, min 8M qlimit 1024
  [ pkts:   1279  bytes:1825459  dropped pkts:  0 bytes:  0 ]
  [ qlength:   0/1024 ]
queue std parent rootq bandwidth 3M, min 2M default qlimit 1024
  [ pkts:   8994  bytes:   12333179  dropped pkts:  0 bytes:  0 ]
  [ qlength:   2/1024 ]

Even after an extended period of execution, I see similar results. The 
supposedly prioritized host sees upload speeds of 17-200KB/s, whereas other 
hosts see 800KB/s or more.

I do not understand the behavior I am seeing. Why are other hosts being 
allocated so much bandwidth for uploads? 

Also of interest is that when I added the queues, a host that reliably used to 
have consistent 27MB/s downloads now sees variable speeds between 13 and 
24MB/s, even when there is no other (meaningful) network activity. I'm not sure 
if this is related to the upload speed issue I am seeing. I realize there is 
outbound control traffic from the downloading host, but I can't imagine that 
should be impacted by the queues when there is no other meaningful network 
traffic. To try to address the download issue, I've experimented with adding a 
control traffic queue and assigning traffic to (std, ctrl), but that hasn't 
helped (in fact, it's hurt).

Based on some past threads I've read on related issues, I've tried adding "max" 
specifications to each queue, but that hasn't helped, and it doesn't seem it 
should be necessary based on the docs. Oddly, if I specify a max of 13 on each 
rule -- with no suffix, which I accidentally did -- I seem to get the desired 
behavior, but in that case pf obviously isn't enforcing the max correctly, and 
I also see download speeds of less than 1KB/s. Adding the intended suffix gives 
the same observable behavior as I saw without the max specifier at all. 

I am running up-to-date 6.9 on ESX 6.7 with vmxnet3 vNICs. The VM has 2 vCPUs 
and 1G and is showing no sign of resource constraints.

Any help or thoughts would be appreciated!

Re: unexpected behavior

[Stuart Henderson]

On 2020-06-09, Sonic  wrote:
> Was wondering if I wanted such an interface for management purposes,
> that is - unconnected during normal installed operation but accepting
> dhcp assignment when connected - could it be placed in a different
> domain (not r0)? This way it should be available when needed but yet
> not interfere with the r0 routing table, that is /etc/mygate would be
> valid and used for the interfaces in r0. Can a new routing domain work
> via dhcp or must its network be defined when it is created?
>
>

Yes.

Re: unexpected behavior

[Sonic]

Was wondering if I wanted such an interface for management purposes,
that is - unconnected during normal installed operation but accepting
dhcp assignment when connected - could it be placed in a different
domain (not r0)? This way it should be available when needed but yet
not interfere with the r0 routing table, that is /etc/mygate would be
valid and used for the interfaces in r0. Can a new routing domain work
via dhcp or must its network be defined when it is created?

Re: unexpected behavior

[Sonic]

Thanks!

On Tue, Jun 2, 2020 at 10:48 AM Anders Andersson  wrote:
>
> On Tue, Jun 2, 2020 at 4:30 PM Sonic  wrote:
> >
> > Recently discovered (snapshot form May 30) having any hostname.if
> > configured for dhcp, even if unplugged and inactive, prevents the
> > default gateway defined in /etc/mygate from being set. Is this normal?
>
> "/etc/mygate is processed after all interfaces have been configured.
> If any hostname.if(5) files contain "dhcp" directives, IPv4 entries in
> /etc/mygate will be ignored.  If they contain "autoconf" directives,
> IPv6 entries will be ignored."
>
> Exactly as documented.

Re: unexpected behavior

[Anders Andersson]

On Tue, Jun 2, 2020 at 4:30 PM Sonic  wrote:
>
> Recently discovered (snapshot form May 30) having any hostname.if
> configured for dhcp, even if unplugged and inactive, prevents the
> default gateway defined in /etc/mygate from being set. Is this normal?

"/etc/mygate is processed after all interfaces have been configured.
If any hostname.if(5) files contain "dhcp" directives, IPv4 entries in
/etc/mygate will be ignored.  If they contain "autoconf" directives,
IPv6 entries will be ignored."

Exactly as documented.

unexpected behavior

[Sonic]

Recently discovered (snapshot form May 30) having any hostname.if
configured for dhcp, even if unplugged and inactive, prevents the
default gateway defined in /etc/mygate from being set. Is this normal?

NAT on enc0 unexpected behavior. Bug? or misconfiguration

[Denis]

I have working IKEv2 VPN between both OpenBSD 6.6 server and 6.6 client
which connects to server from behind ISP NAT. My configuration is very
close to FAQ with OpenBSD Client connection behind ISP NAT to a server:
https://www.openbsd.org/faq/faq17.html#clientikev2

When VPN is disconnected, NAT on egress works as should be for client
itself and LAN hosts connected to the client using pf.conf second NAT rule:

...
match out log on enc0 inet all nat-to 10.0.5.2 tagged WEB
match out log on egress from 192.168.2.0/24 to any nat-to (egress)
tagged WEB
...

Once VPN is connected, NAT works for client itself only, and no NAT for
client's LAN connected hosts on enc0 using first rule above.
For instance, there is no NAT on enc0 from VMM host 192.168.2.4 from
virtual LAN 192.168.2.0/24. The same is for physical LAN connected hosts
to client machine.

$ telnet 172.217.21.142 80 (from LAN VMM host 192.168.2.4)

tcpdump -en -i pflog0
13.29.33.694116 rule 4/(match) match out on enc0: 10.0.5.2.64401 >
172.217.21.142.80 S 3601041753:3601041753(0) win 64240  [tos 0x10]
13.29.33.694116 rule 135/(match) match out on enc0: 10.0.5.2.64401 >
172.217.21.142.80 S 3601041753:3601041753(0) win 64240  [tos 0x10]
13.29.34.316393 rule 4/(match) match out on enc0: 10.0.5.2.50426 >
192.168.2.4.59062 S 880722202:880722202(0) ack 3601041754 win 60192 
13.29.34.625518 rule 4/(match) match out on enc0: 10.0.5.2.54501 >
192.168.2.4.59062 S 880722202:880722202(0) ack 3601041754 win 60192 


Initiator's VMM LAN SA bypassed in /etc/ipsec.conf in all directions
flow from 192.168.2.0/24 to 192.168.2.0/24 type bypass
flow from 127.0.0.1/32 to 192.168.2.0/24 type bypass
flow from 192.168.2.0/24 to 127.0.0.1/32 type bypass

responder /etc/iked.conf
ikev2 'responder' passive esp \
from 0.0.0.0/0 to 10.0.5.0/24 \
local 9.8.7.6 peer any \
srcid srv.vpn \
tag "ROADW"

initiator /etc/iked.conf
ikev2 'initiator' active esp \
from 10.0.5.2 (0.0.0.0/0) to 0.0.0.0/0 \ => to have traffic
appears for LAN hosts from 10.0.5.2 as in IPSEC.CONF(5) for NAT
configurations
peer 9.8.7.6 \
srcid clnt.vpn \
dstid srv.vpn

/etc/pf.conf (client)
...
match out log on enc0 inet all nat-to 10.0.5.2 tagged WEB
...

pass in log on enc0 inet from 0.0.0.0/0 to 0.0.0.0/0 keep state (if-bound)
pass out log on enc0 inet from 0.0.0.0/0 to 0.0.0.0/0 keep state
(if-bound) tagged WEB

...
pass in on 192.168.2.1 inet proto tcp from 192.168.2.0/24 to any port
{www, https} flags S/SA modulate state tag WEB

Re: unexpected behavior with static route inserted

[Sebastian Benoit]

Benjamin Girard(benjamin.gir...@kambi.com) on 2019.08.22 12:35:08 +:
> Hi,
> 
> 
> I have the following machine with two interfaces like this:
> 
> root@fw:~ # cat /etc/hostname.vlan10
> vlan 10 vlandev vio0

shoudl be 'vnetid 10 parent vio0'

> inet 10.0.0.1 255.255.255.0 NONE
> up
> 
> root@fw:~ # cat /etc/hostname.vlan23
> vlan 23 vlandev vio0

should be 'vnetid 23 parent vio0'

> inet 172.30.133.83 255.255.255.240 NONE
> !route add 10.0.0.10/32 172.30.133.84
> up
> 
> now the problem is that if i start pinging 10.0.0.10 before my vlan23 is up 
> and has inserted the static route, the openbsd machine will insert another 
> route like this:
> 
> root@fw:~ $ route -n show | grep 10.0.0.10
> 10.0.0.10link#12UHLc   0   78 - 3 vlan10
> 
> then if I netstart vlan23 the static route will be inserted but with a lower 
> priority:
> 
> kmbops@fw:~ $ route -n show | grep 209.43.38.6
> 10.0.0.10link#12UHLc   0   78 - 3 vlan10
> 10.0.0.10172.30.133.84  UGHS   00 - 8 vlan23
> 
> Eventually this first route inserted will disappear if i stop pinging the
> ip, or will not at all exist in the first place if i don't try to reach
> that ip before inserting the static route, but my question is: Is it the
> expected behavior?

Yes.

> Why does this route get inserted while there is already a route for the
> full 10.0.0.0/24 subnet existing anyway?

Simplified, this is how the network stack stores the information that it is
trying to find the link-local address of 10.0.0.10 on that link (by sending
arp requests).

If you run OpenBSD 6.5 or newer, you can try to solve your problem by
renaming the vlan23 hostname.if file:

 mv /etc/hostname.vlan23 /etc/hostname.vlan1

Then it will be created before vlan10, and things should just work.

unexpected behavior with static route inserted

[Benjamin Girard]

Hi,


I have the following machine with two interfaces like this:

root@fw:~ # cat /etc/hostname.vlan10
vlan 10 vlandev vio0
inet 10.0.0.1 255.255.255.0 NONE
up

root@fw:~ # cat /etc/hostname.vlan23
vlan 23 vlandev vio0
inet 172.30.133.83 255.255.255.240 NONE
!route add 10.0.0.10/32 172.30.133.84
up

now the problem is that if i start pinging 10.0.0.10 before my vlan23 is up and 
has inserted the static route, the openbsd machine will insert another route 
like this:

root@fw:~ $ route -n show | grep 10.0.0.10
10.0.0.10link#12UHLc   0   78 - 3 vlan10

then if I netstart vlan23 the static route will be inserted but with a lower 
priority:

kmbops@fw:~ $ route -n show | grep 209.43.38.6
10.0.0.10link#12UHLc   0   78 - 3 vlan10
10.0.0.10172.30.133.84  UGHS   00 - 8 vlan23

Eventually this first route inserted will disappear if i stop pinging the ip, 
or will not at all exist in the first place if i don't try to reach that ip 
before inserting the static route, but my question is: Is it the expected 
behavior?

Why does this route get inserted while there is already a route for the full 
10.0.0.0/24 subnet existing anyway?

Thanks,
Ben

Re: unexpected behavior

[Edgar Pettijohn]

On 05/15/17 19:18, Ingo Schwarze wrote:

Hi Edgar,

Edgar Pettijohn wrote on Mon, May 15, 2017 at 06:57:57PM -0500:


I think this may be normal, but I can't seem to find it documented anywhere.
While viewing a man page in xterm if I hit the 'v' key it appears to open it
for editing in vi. I did so accidentally recently and at first I thought I
broke something because of all the "F^HFI^HIL^HLE^,etc" which I assume are
man markup characters. I looked around for documentation on this "feature",
but I'm not sure where to look vi, mandoc, xterm? So is this a bug somewhere
or expected behavior?

It may be less(1) expected than you think, but it is.

Actually editing the file is not very useful because it's a tmp file.
To get out of vi(1) again, just use the normal ":q".

For documentation, see

   http://man.openbsd.org/less#v

or type "man less", then hit the keys ENTER, COLON, t, v, ENTER.

If you don't like all that, you can set

  $ export LESSEDIT=false

Yours,
   Ingo
Thanks to Ingo and Mike.  After reading your replies I vaguely recall 
reading that in the past.

Re: unexpected behavior

[Ingo Schwarze]

Hi Edgar,

Edgar Pettijohn wrote on Mon, May 15, 2017 at 06:57:57PM -0500:

> I think this may be normal, but I can't seem to find it documented anywhere.
> While viewing a man page in xterm if I hit the 'v' key it appears to open it
> for editing in vi. I did so accidentally recently and at first I thought I
> broke something because of all the "F^HFI^HIL^HLE^,etc" which I assume are
> man markup characters. I looked around for documentation on this "feature",
> but I'm not sure where to look vi, mandoc, xterm? So is this a bug somewhere
> or expected behavior?

It may be less(1) expected than you think, but it is.

Actually editing the file is not very useful because it's a tmp file.
To get out of vi(1) again, just use the normal ":q".

For documentation, see

  http://man.openbsd.org/less#v

or type "man less", then hit the keys ENTER, COLON, t, v, ENTER.

If you don't like all that, you can set

 $ export LESSEDIT=false

Yours,
  Ingo

Re: unexpected behavior

[Mike Burns]

On 2017-05-15 18.57.57 -0500, Edgar Pettijohn wrote:
> I think this may be normal, but I can't seem to find it documented anywhere.
> While viewing a man page in xterm if I hit the 'v' key it appears to open it
> for editing in vi. I did so accidentally recently and at first I thought I
> broke something because of all the "F^HFI^HIL^HLE^,etc" which I assume are
> man markup characters. I looked around for documentation on this "feature",
> but I'm not sure where to look vi, mandoc, xterm? So is this a bug somewhere
> or expected behavior?

less(1)

  _v_
  Invokes an editor to edit the current file being viewed. The editor is
  taken from the environment variable VISUAL, if defined, or EDITOR if
  VISUAL is not defined, or defaults to "vi" if neither VISUAL nor
  EDITOR is defined.

http://man.openbsd.org/less#v

unexpected behavior

[Edgar Pettijohn]

I think this may be normal, but I can't seem to find it documented 
anywhere.  While viewing a man page in xterm if I hit the 'v' key it 
appears to open it for editing in vi. I did so accidentally recently and 
at first I thought I broke something because of all the 
"F^HFI^HIL^HLE^,etc" which I assume are man markup characters. I looked 
around for documentation on this "feature", but I'm not sure where to 
look vi, mandoc, xterm? So is this a bug somewhere or expected behavior?


Thanks,


Edgar

Re: Unexpected behavior in su/doas

[Tinker]

On 2016-10-02 18:14, Chris Bennett wrote:

On Sun, Oct 02, 2016 at 01:03:28AM -0700, Philip Guenther wrote:

On Sun, Oct 2, 2016 at 12:35 AM, Otto Moerbeek  wrote:
> On Sat, Oct 01, 2016 at 05:15:31PM -0500, Chris Bennett wrote:
>
>> On Sat, Oct 01, 2016 at 03:54:40PM -0600, Theo de Raadt wrote:
>> > Use of su, doas, or sudo -- means you EXPLICITLY want the tty to
>> > remain the same.
>> >
>> > De-escalation using these "sudo" or "doas" like tools on a tty is
>> > somewhat unsafe - it has always been unsafe - because tty's have
>> > capabilities.
>> >
>> > If you wish to be safer, do these operations without retaining access
>> > to a tty.
>> >
>> > Escalation on the other hand (user -> root) is different, because then
>> > it is clear you want to do more / everything.  But de-escalation is a
>> > joke.
>> >
>> > This is just one mechanism on tty, there are others.  On other
>> > descriptors there are other abilities.
>> >
>>
>> Would you mind explaining this a little bit. I don't really mean the
>> sudo/doas part.
>>
>> How to do operations without retaining access to a tty?
>>
>> What other descriptors?
>
> Well, a lot of things are possible using descriptors. Descriptors can
> refer to files, devices, sockets to name a few. So if you have an open
> descriptor to any of them...

...and it's not just actual file descriptors that provide privileged
access: even if a process closes all fds for its controlling tty, it
remains the process's controlling tty and can still be reopened via
/dev/tty.  Similarly, simply being in the same session gives a process
additional rights that it wouldn't have otherwise, such as being able
to use tcsetpgrp() and see your login name via getlogin()...



So fork, as used in daemon does mitigate this, as long as used 
correctly?

Or does the same/other problems continue?

Chris Bennett


Doing "su" or "chroot" (symmetric with respect to this risk) is 
sometimes very convenient, as it can be done without external 
configuration as would be needed for ssh.


Would it be possible to cut the risk (file descriptors) while still 
outputting to the same terminal, ssh without ssh?

Re: Unexpected behavior in su/doas

[Chris Bennett]

On Sun, Oct 02, 2016 at 01:03:28AM -0700, Philip Guenther wrote:
> On Sun, Oct 2, 2016 at 12:35 AM, Otto Moerbeek  wrote:
> > On Sat, Oct 01, 2016 at 05:15:31PM -0500, Chris Bennett wrote:
> >
> >> On Sat, Oct 01, 2016 at 03:54:40PM -0600, Theo de Raadt wrote:
> >> > Use of su, doas, or sudo -- means you EXPLICITLY want the tty to
> >> > remain the same.
> >> >
> >> > De-escalation using these "sudo" or "doas" like tools on a tty is
> >> > somewhat unsafe - it has always been unsafe - because tty's have
> >> > capabilities.
> >> >
> >> > If you wish to be safer, do these operations without retaining access
> >> > to a tty.
> >> >
> >> > Escalation on the other hand (user -> root) is different, because then
> >> > it is clear you want to do more / everything.  But de-escalation is a
> >> > joke.
> >> >
> >> > This is just one mechanism on tty, there are others.  On other
> >> > descriptors there are other abilities.
> >> >
> >>
> >> Would you mind explaining this a little bit. I don't really mean the
> >> sudo/doas part.
> >>
> >> How to do operations without retaining access to a tty?
> >>
> >> What other descriptors?
> >
> > Well, a lot of things are possible using descriptors. Descriptors can
> > refer to files, devices, sockets to name a few. So if you have an open
> > descriptor to any of them...
> 
> ...and it's not just actual file descriptors that provide privileged
> access: even if a process closes all fds for its controlling tty, it
> remains the process's controlling tty and can still be reopened via
> /dev/tty.  Similarly, simply being in the same session gives a process
> additional rights that it wouldn't have otherwise, such as being able
> to use tcsetpgrp() and see your login name via getlogin()...
> 

So fork, as used in daemon does mitigate this, as long as used correctly?
Or does the same/other problems continue?

Chris Bennett

Re: Unexpected behavior in su/doas

[Lampshade]

> > This is just one mechanism on tty, there are others.  On other
> > descriptors there are other abilities.
> > 
> 
> Would you mind explaining this a little bit. I don't really mean the
> sudo/doas part.
> 
> How to do operations without retaining access to a tty?
>
> What other descriptors?

Example:
If you have file descriptor to directory outside chroot
and you are root user you can escape chroot.

https://filippo.io/escaping-a-chroot-jail-slash-1/

Re: Unexpected behavior in su/doas

[Philip Guenther]

On Sun, Oct 2, 2016 at 12:35 AM, Otto Moerbeek  wrote:
> On Sat, Oct 01, 2016 at 05:15:31PM -0500, Chris Bennett wrote:
>
>> On Sat, Oct 01, 2016 at 03:54:40PM -0600, Theo de Raadt wrote:
>> > Use of su, doas, or sudo -- means you EXPLICITLY want the tty to
>> > remain the same.
>> >
>> > De-escalation using these "sudo" or "doas" like tools on a tty is
>> > somewhat unsafe - it has always been unsafe - because tty's have
>> > capabilities.
>> >
>> > If you wish to be safer, do these operations without retaining access
>> > to a tty.
>> >
>> > Escalation on the other hand (user -> root) is different, because then
>> > it is clear you want to do more / everything.  But de-escalation is a
>> > joke.
>> >
>> > This is just one mechanism on tty, there are others.  On other
>> > descriptors there are other abilities.
>> >
>>
>> Would you mind explaining this a little bit. I don't really mean the
>> sudo/doas part.
>>
>> How to do operations without retaining access to a tty?
>>
>> What other descriptors?
>
> Well, a lot of things are possible using descriptors. Descriptors can
> refer to files, devices, sockets to name a few. So if you have an open
> descriptor to any of them...

...and it's not just actual file descriptors that provide privileged
access: even if a process closes all fds for its controlling tty, it
remains the process's controlling tty and can still be reopened via
/dev/tty.  Similarly, simply being in the same session gives a process
additional rights that it wouldn't have otherwise, such as being able
to use tcsetpgrp() and see your login name via getlogin()...


Philip Guenther

Re: Unexpected behavior in su/doas

[Otto Moerbeek]

On Sat, Oct 01, 2016 at 05:15:31PM -0500, Chris Bennett wrote:

> On Sat, Oct 01, 2016 at 03:54:40PM -0600, Theo de Raadt wrote:
> > Use of su, doas, or sudo -- means you EXPLICITLY want the tty to
> > remain the same.
> > 
> > De-escalation using these "sudo" or "doas" like tools on a tty is
> > somewhat unsafe - it has always been unsafe - because tty's have
> > capabilities.
> > 
> > If you wish to be safer, do these operations without retaining access
> > to a tty.
> > 
> > Escalation on the other hand (user -> root) is different, because then
> > it is clear you want to do more / everything.  But de-escalation is a
> > joke.
> > 
> > This is just one mechanism on tty, there are others.  On other
> > descriptors there are other abilities.
> > 
> 
> Would you mind explaining this a little bit. I don't really mean the
> sudo/doas part.
> 
> How to do operations without retaining access to a tty?
>
> What other descriptors?

Well, a lot of things are possible using descriptors. Descriptors can
refer to files, devices, sockets to name a few. So if you have an open
descriptor to any of them...

> 
> And, I would especially appreciate any areas in src that could more
> fully give me an understanding of this. Studying code has to be
> essential to get this.

e.g. login(1), cron(8), daemon(3) and setsid(2) and friends.

-Otto

Re: Unexpected behavior in su/doas

[Otto Moerbeek]

On Sun, Oct 02, 2016 at 07:10:12AM +0200, Sebastien Marie wrote:

> On Sat, Oct 01, 2016 at 05:15:31PM -0500, Chris Bennett wrote:
> > On Sat, Oct 01, 2016 at 03:54:40PM -0600, Theo de Raadt wrote:
> > > Use of su, doas, or sudo -- means you EXPLICITLY want the tty to
> > > remain the same.
> > > 
> > > De-escalation using these "sudo" or "doas" like tools on a tty is
> > > somewhat unsafe - it has always been unsafe - because tty's have
> > > capabilities.
> > > 
> > > If you wish to be safer, do these operations without retaining access
> > > to a tty.
> > > 
> > > Escalation on the other hand (user -> root) is different, because then
> > > it is clear you want to do more / everything.  But de-escalation is a
> > > joke.
> > > 
> > > This is just one mechanism on tty, there are others.  On other
> > > descriptors there are other abilities.
> > > 
> > 
> > Would you mind explaining this a little bit. I don't really mean the
> > sudo/doas part.
> > 
> > How to do operations without retaining access to a tty?
> > What other descriptors?
> > 
> > And, I would especially appreciate any areas in src that could more
> > fully give me an understanding of this. Studying code has to be
> > essential to get this.
> > 
> 
> there is a recent thread on oss-security about the specific problem of
> sharing tty:
> http://openwall.com/lists/oss-security/2016/09/25/1
> 
> or a more ancien stuff (same problem): 
> http://www.openwall.com/lists/oss-security/2011/12/20/2
> 
> when using doas/sudo you share a tty.
> 
> # tty
> /dev/ttypa
> # doas -u user -s
> $ tty
> /dev/ttypa
> 
> so at a moment, user has access to the tty device that root will use
> later.
> 
> for example, user is able to push chars in tty buffer, logout, and let
> the root process eats the controlled input.
> 
> alternative way (and more secure in this context) is to use ssh(1). But
> note it needs additionnal configuration. ssh(1) will allocate a new
> pty(4) device for the user.
> 
> # tty
> /dev/ttypa
> # ssh user@localhost
> Last login: ...
> OpenBSD 6.0-current ...
> ...
> 
> $ tty
> /dev/ttypb
> 
> Regards.
> -- 
> Sebastien Marie

Alternatively, you can run a command as a specific use without tty
using batch(1) or at(1).

# echo su otto -c "id; tty" | batch

The command will be executed without controlling tty. OUtput will be
reported by mail.

-Otto

Re: Unexpected behavior in su/doas

[Sebastien Marie]

On Sat, Oct 01, 2016 at 05:15:31PM -0500, Chris Bennett wrote:
> On Sat, Oct 01, 2016 at 03:54:40PM -0600, Theo de Raadt wrote:
> > Use of su, doas, or sudo -- means you EXPLICITLY want the tty to
> > remain the same.
> > 
> > De-escalation using these "sudo" or "doas" like tools on a tty is
> > somewhat unsafe - it has always been unsafe - because tty's have
> > capabilities.
> > 
> > If you wish to be safer, do these operations without retaining access
> > to a tty.
> > 
> > Escalation on the other hand (user -> root) is different, because then
> > it is clear you want to do more / everything.  But de-escalation is a
> > joke.
> > 
> > This is just one mechanism on tty, there are others.  On other
> > descriptors there are other abilities.
> > 
> 
> Would you mind explaining this a little bit. I don't really mean the
> sudo/doas part.
> 
> How to do operations without retaining access to a tty?
> What other descriptors?
> 
> And, I would especially appreciate any areas in src that could more
> fully give me an understanding of this. Studying code has to be
> essential to get this.
> 

there is a recent thread on oss-security about the specific problem of
sharing tty:
http://openwall.com/lists/oss-security/2016/09/25/1

or a more ancien stuff (same problem): 
http://www.openwall.com/lists/oss-security/2011/12/20/2

when using doas/sudo you share a tty.

# tty
/dev/ttypa
# doas -u user -s
$ tty
/dev/ttypa

so at a moment, user has access to the tty device that root will use
later.

for example, user is able to push chars in tty buffer, logout, and let
the root process eats the controlled input.

alternative way (and more secure in this context) is to use ssh(1). But
note it needs additionnal configuration. ssh(1) will allocate a new
pty(4) device for the user.

# tty
/dev/ttypa
# ssh user@localhost
Last login: ...
OpenBSD 6.0-current ...
...

$ tty
/dev/ttypb

Regards.
-- 
Sebastien Marie

Re: Unexpected behavior in su/doas

[Chris Bennett]

On Sat, Oct 01, 2016 at 03:54:40PM -0600, Theo de Raadt wrote:
> Use of su, doas, or sudo -- means you EXPLICITLY want the tty to
> remain the same.
> 
> De-escalation using these "sudo" or "doas" like tools on a tty is
> somewhat unsafe - it has always been unsafe - because tty's have
> capabilities.
> 
> If you wish to be safer, do these operations without retaining access
> to a tty.
> 
> Escalation on the other hand (user -> root) is different, because then
> it is clear you want to do more / everything.  But de-escalation is a
> joke.
> 
> This is just one mechanism on tty, there are others.  On other
> descriptors there are other abilities.
> 

Would you mind explaining this a little bit. I don't really mean the
sudo/doas part.

How to do operations without retaining access to a tty?
What other descriptors?

And, I would especially appreciate any areas in src that could more
fully give me an understanding of this. Studying code has to be
essential to get this.

Thank you very much,
Chris Bennett

Unexpected behavior of netstat in 4.3

[Andreas Maus]

Hi.

I noticed the following misbehavior of netstat in 4.3.

The manual page says:

[... snipp ...]
 -I interface
Show information about the specified interface; used with a wait
interval as described below.
[... snipp ...]

O.K.

Using 4.2 netstat -I interface works as expected:

-bash-3.2$ uname -a 
OpenBSD piglet.badphish.dyndns.org 4.2 GENERIC#0 i386
-bash-3.2$ netstat -ib -I fxp0
NameMtu   Network Address   Ibytes Obytes
fxp01500  Link  00:a0:c9:a0:73:84  514885834  263525662
fxp01500  fe80::%fxp0 fe80::2a0:c9ff:fe  514885834  263525662

But using the same command in 4.3 returns:

-bash-3.2$ uname -a 
OpenBSD son-goku.badphish.dyndns.org 4.3 GENERIC#698 i386
-bash-3.2$ netstat -ib -I vr0 
NameMtu   Network Address   Ibytes Obytes
lo0 33208 localhost   localhost.badphis   37219614   37219614
lo0 33208 localhost.b localhost.badphis   37219614   37219614
lo0 33208 fe80::%lo0/ fe80::1%lo0 37219614   37219614
vr0 1500  Link  00:00:24:ca:68:38 26230376471 6788235902
vr0 1500  son-goku-un son-goku-untruste 26230376471 6788235902
vr0 1500  fe80::%vr0/ fe80::200:24ff:fe 26230376471 6788235902
vr1 1500  son-goku-dm son-goku-dmz.badp 21983028148 89288229482
vr1 1500  fe80::%vr1/ fe80::200:24ff:fe 21983028148 89288229482
vr2 1500  son-goku-tr son-goku-trusted. 93517695317 47526180347
vr2 1500  fe80::%vr2/ fe80::200:24ff:fe 93517695317 47526180347
vr3 1500  192.168.211 192.168.211.1  174328335  863529211
vr3 1500  fe80::%vr3/ fe80::200:24ff:fe  174328335  863529211

At least list of changes (http://openbsd.org/plus43.html) notes some
modifications to netstat but not this behavior.

Maybe a bug? Or how do I request the information for just a single
interface?

So long,

Andreas.

-- 
Windows 95: A 32-bit patch for a 16-bit GUI shell running on top of
an 8-bit operating system written for a 4-bit processor by a 2-bit
company who cannot stand 1 bit of competition.

Re: Unexpected behavior of netstat in 4.3

[Jason McIntyre]

On Sat, Jun 21, 2008 at 09:30:36AM +0200, Andreas Maus wrote:
 Hi.
 
 I noticed the following misbehavior of netstat in 4.3.
 
 The manual page says:
 
 [... snipp ...]
  -I interface
 Show information about the specified interface; used with a wait
 interval as described below.
 [... snipp ...]
 
 O.K.
 
 Using 4.2 netstat -I interface works as expected:
 
 -bash-3.2$ uname -a 
 OpenBSD piglet.badphish.dyndns.org 4.2 GENERIC#0 i386
 -bash-3.2$ netstat -ib -I fxp0
 NameMtu   Network Address   Ibytes Obytes
 fxp01500  Link  00:a0:c9:a0:73:84  514885834  263525662
 fxp01500  fe80::%fxp0 fe80::2a0:c9ff:fe  514885834  263525662
 
 But using the same command in 4.3 returns:
 
 -bash-3.2$ uname -a 
 OpenBSD son-goku.badphish.dyndns.org 4.3 GENERIC#698 i386
 -bash-3.2$ netstat -ib -I vr0 
 NameMtu   Network Address   Ibytes Obytes
 lo0 33208 localhost   localhost.badphis   37219614   37219614
 lo0 33208 localhost.b localhost.badphis   37219614   37219614
 lo0 33208 fe80::%lo0/ fe80::1%lo0 37219614   37219614
 vr0 1500  Link  00:00:24:ca:68:38 26230376471 6788235902
 vr0 1500  son-goku-un son-goku-untruste 26230376471 6788235902
 vr0 1500  fe80::%vr0/ fe80::200:24ff:fe 26230376471 6788235902
 vr1 1500  son-goku-dm son-goku-dmz.badp 21983028148 89288229482
 vr1 1500  fe80::%vr1/ fe80::200:24ff:fe 21983028148 89288229482
 vr2 1500  son-goku-tr son-goku-trusted. 93517695317 47526180347
 vr2 1500  fe80::%vr2/ fe80::200:24ff:fe 93517695317 47526180347
 vr3 1500  192.168.211 192.168.211.1  174328335  863529211
 vr3 1500  fe80::%vr3/ fe80::200:24ff:fe  174328335  863529211
 
 At least list of changes (http://openbsd.org/plus43.html) notes some
 modifications to netstat but not this behavior.
 
 Maybe a bug? Or how do I request the information for just a single
 interface?
 
 So long,
 
 Andreas.
 

this was fixed here:


revision 1.57
date: 2008/03/18 20:03:37;  author: claudio;  state: Exp;  lines: +4 -1
Make -I work correctly. RTM_NEWADDR needs to filter the ifname as well.


jmc

Re: Unexpected behavior of netstat in 4.3

[Jason McIntyre]

On Sat, Jun 21, 2008 at 09:07:14AM +0100, Jason McIntyre wrote:
 
 this was fixed here:
 
 
 revision 1.57
 date: 2008/03/18 20:03:37;  author: claudio;  state: Exp;  lines: +4 -1
 Make -I work correctly. RTM_NEWADDR needs to filter the ifname as well.
 
 

sorry, should have said this was from netstat/if.c.
jmc

Re: Unexpected behavior of netstat in 4.3

[Andreas Maus]

On Sat, Jun 21, 2008 at 09:07:14AM +0100, Jason McIntyre wrote:
 this was fixed here:
 
 
 revision 1.57
 date: 2008/03/18 20:03:37;  author: claudio;  state: Exp;  lines: +4 -1
 Make -I work correctly. RTM_NEWADDR needs to filter the ifname as well.
 
Aha! Known issue ;)

Well I will extract a diff and recompile only netstat.
(I'm not comfortable rebuilding everything on a 500MHz
server ...)

Thanks for your help.

Andreas.

-- 
Windows 95: A 32-bit patch for a 16-bit GUI shell running on top of
an 8-bit operating system written for a 4-bit processor by a 2-bit
company who cannot stand 1 bit of competition.