Re: RLIMIT_CPU doesn't work reliably on mostly idle systems

[Scott Cheloha]

> On Aug 28, 2023, at 20:04, Eric Wong  wrote:
> 
> 
>> 
>> Synopsis: RLIMIT_CPU doesn't work reliably on mostly idle systems
>> Category: system
>> Environment:
>System  : OpenBSD 7.3
>Details : OpenBSD 7.3 (GENERIC.MP) #1242: Sat Mar 25 18:04:31 MDT 2023
> 
> dera...@octeon.openbsd.org:/usr/src/sys/arch/octeon/compile/GENERIC.MP
> 
>Architecture: OpenBSD.octeon
>Machine : octeon
>> Description:
> 
> RLIMIT_CPU doesn't work reliably when few/no syscalls are made on an
> otherwise idle system (aside from the test process using up CPU).
> It can take 20-50s to fire SIGKILL with rlim_max=9 (and the SIGXCPU
> from rlim_cur=1 won't even fire).
> 
> I can reproduce this on a private amd64 VM and also on gcc231
> on GCC compiler farm .
> I can't reproduce this on a busy system like gcc220 on cfarm,
> however.
> 
>> How-To-Repeat:
> 
> Following is a standalone C program which demonstrates the problem on
> a mostly idle system:
> /*
> * Most reliably reproduced with compiler optimizations disabled:
> *
> *   cc -o rlimit_cpu -ggdb3 -Wall rlimit_cpu.c
> *
> * Neither SIGXCPU (from rlim_cur) nor SIGKILL (from rlim_max)
> * with RLIMIT_CPU set seem to fire reliably with few syscalls being made.
> * On an otherwise idle system, it can take many more seconds (20-50s)
> * than expected when rlim_max=9 (SIGXCPU doesn't even happen).
> * Best case is 2 seconds for SIGXCPU when rlim_cur=1 on a busy system
> * which is understandable due to kernel accounting delays.
> *
> * I rely on RLIMIT_CPU to protect systems from pathological userspace
> * code (diff generation, regexps, etc)
> *
> * Testing on cfarm  machines,
> * the issue is visible on a mostly-idle 4-core gcc231 mips64
> * but doesn't seem to happen on the busy 12-core gcc220 machine
> * (only 2 seconds for XCPU w/ rlim_cur=1)
> */
> #include 
> #include 
> #include 
> #include 
> 
> static void sigxcpu(int sig)
> {
>write(1, "SIGXCPU\n", 8);
>_exit(1);
> }
> 
> static volatile size_t nr; // volatile to disable optimizations
> int main(void)
> {
>struct rlimit rlim = { .rlim_cur = 1, .rlim_max = 9 };
>int rc;
> 
>signal(SIGXCPU, sigxcpu);
>rc = setrlimit(RLIMIT_CPU, );
>assert(rc == 0);
> 
>/*
> * adding some time, times, and write syscalls improve likelyhood of
> * of rlimit signals firing in a timely manner.  writes to /dev/null
> * seems less-likely to trigger than writes to the terminal or
> * regular file.
> */
>for (;; nr++) {
>}
> 
>return 0;
> }
> 
>> Fix:
>Making more syscalls can workaround the problem, but that's not
> an option when dealing with userspace-heavy code like pathological regexps.

The CPU time limit is checked from a
periodic timeout.

CPU time totals accumulate in mi_switch().

The problem is that on a mostly idle system,
a user thread that is hogging the CPU may
take a very long time to switch out, and the
and all that CPU time doesn't accumulate
until the switch, and so the signal will
arrive later than requested.

System calls have points where a thread can
switch out, which is why the delay is
exaggerated on synthetic workloads like the
busy-loop shown above.

One possible solution is to check usage
times for threads that are still ONPROC
during the rusage timeout.

Another approach is to be more aggressive
about forcing threads to switch out, even
when nothing else wants to run.

Coincidentally, we are discussing p_rtime on
tech@ right now, which is tangentially related
to this issue.

RLIMIT_CPU doesn't work reliably on mostly idle systems

[Eric Wong]

>Synopsis: RLIMIT_CPU doesn't work reliably on mostly idle systems
>Category: system
>Environment:
System  : OpenBSD 7.3
Details : OpenBSD 7.3 (GENERIC.MP) #1242: Sat Mar 25 18:04:31 MDT 
2023
 
dera...@octeon.openbsd.org:/usr/src/sys/arch/octeon/compile/GENERIC.MP

Architecture: OpenBSD.octeon
Machine : octeon
>Description:

RLIMIT_CPU doesn't work reliably when few/no syscalls are made on an
otherwise idle system (aside from the test process using up CPU).
It can take 20-50s to fire SIGKILL with rlim_max=9 (and the SIGXCPU
from rlim_cur=1 won't even fire).

I can reproduce this on a private amd64 VM and also on gcc231
on GCC compiler farm .
I can't reproduce this on a busy system like gcc220 on cfarm,
however.

>How-To-Repeat:

Following is a standalone C program which demonstrates the problem on
a mostly idle system:
/*
 * Most reliably reproduced with compiler optimizations disabled:
 *
 *   cc -o rlimit_cpu -ggdb3 -Wall rlimit_cpu.c
 *
 * Neither SIGXCPU (from rlim_cur) nor SIGKILL (from rlim_max)
 * with RLIMIT_CPU set seem to fire reliably with few syscalls being made.
 * On an otherwise idle system, it can take many more seconds (20-50s)
 * than expected when rlim_max=9 (SIGXCPU doesn't even happen).
 * Best case is 2 seconds for SIGXCPU when rlim_cur=1 on a busy system
 * which is understandable due to kernel accounting delays.
 *
 * I rely on RLIMIT_CPU to protect systems from pathological userspace
 * code (diff generation, regexps, etc)
 *
 * Testing on cfarm  machines,
 * the issue is visible on a mostly-idle 4-core gcc231 mips64
 * but doesn't seem to happen on the busy 12-core gcc220 machine
 * (only 2 seconds for XCPU w/ rlim_cur=1)
 */
#include 
#include 
#include 
#include 

static void sigxcpu(int sig)
{
write(1, "SIGXCPU\n", 8);
_exit(1);
}

static volatile size_t nr; // volatile to disable optimizations
int main(void)
{
struct rlimit rlim = { .rlim_cur = 1, .rlim_max = 9 };
int rc;

signal(SIGXCPU, sigxcpu);
rc = setrlimit(RLIMIT_CPU, );
assert(rc == 0);

/*
 * adding some time, times, and write syscalls improve likelyhood of
 * of rlimit signals firing in a timely manner.  writes to /dev/null
 * seems less-likely to trigger than writes to the terminal or
 * regular file.
 */
for (;; nr++) {
}

return 0;
}

>Fix:
Making more syscalls can workaround the problem, but that's not
an option when dealing with userspace-heavy code like pathological regexps.


dmesg:
OpenBSD 7.3 (GENERIC.MP) #1242: Sat Mar 25 18:04:31 MDT 2023
dera...@octeon.openbsd.org:/usr/src/sys/arch/octeon/compile/GENERIC.MP
real mem = 1073741824 (1024MB)
avail mem = 1035304960 (987MB)
random: good seed from bootblocks
mainbus0 at root: board 20300 rev 0.15, model cavium,ubnt_e300
cpu0 at mainbus0: CN70xx/CN71xx CPU rev 0.2 1000 MHz, CN70xx/CN71xx FPU rev 0.0
cpu0: cache L1-I 78KB 39 way D 32KB 32 way, L2 1024KB 8 way
cpu1 at mainbus0: CN70xx/CN71xx CPU rev 0.2 1000 MHz, CN70xx/CN71xx FPU rev 0.0
cpu1: cache L1-I 78KB 39 way D 32KB 32 way, L2 1024KB 8 way
cpu2 at mainbus0: CN70xx/CN71xx CPU rev 0.2 1000 MHz, CN70xx/CN71xx FPU rev 0.0
cpu2: cache L1-I 78KB 39 way D 32KB 32 way, L2 1024KB 8 way
cpu3 at mainbus0: CN70xx/CN71xx CPU rev 0.2 1000 MHz, CN70xx/CN71xx FPU rev 0.0
cpu3: cache L1-I 78KB 39 way D 32KB 32 way, L2 1024KB 8 way
clock0 at mainbus0: int 5
octcrypto0 at mainbus0
iobus0 at mainbus0
simplebus0 at iobus0: "soc"
"bootbus" at simplebus0 not configured
octciu0 at simplebus0
octcib0 at simplebus0: max-bits 23
octcib1 at simplebus0: max-bits 12
octcib2 at simplebus0: max-bits 6
octcib3 at simplebus0: max-bits 15
octcib4 at simplebus0: max-bits 4
octcib5 at simplebus0: max-bits 11
octcib6 at simplebus0: max-bits 11
octgpio0 at simplebus0: 20 pins, xbit 16
octsmi0 at simplebus0
octsmi1 at simplebus0
octpip0 at simplebus0
octgmx0 at octpip0 interface 0
cnmac0 at octgmx0: port 0 SGMII, address e0:63:da:c0:62:27
ukphy0 at cnmac0 phy 4: Generic IEEE 802.3u media interface, rev. 2: OUI 
0x0001c1, model 0x000c
cnmac1 at octgmx0: port 1 SGMII, address e0:63:da:c0:62:28
ukphy1 at cnmac1 phy 5: Generic IEEE 802.3u media interface, rev. 2: OUI 
0x0001c1, model 0x000c
cnmac2 at octgmx0: port 2 SGMII, address e0:63:da:c0:62:29
ukphy2 at cnmac2 phy 6: Generic IEEE 802.3u media interface, rev. 2: OUI 
0x0001c1, model 0x000c
cnmac3 at octgmx0: port 3 SGMII, address e0:63:da:c0:62:2a
ukphy3 at cnmac3 phy 7: Generic IEEE 802.3u media interface, rev. 2: OUI 
0x0001c1, model 0x000c
octsctl0 at simplebus0: disabled
octxctl0 at simplebus0: DWC3 rev 0x250a
xhci0 at octxctl0, xHCI 1.0
usb0 at xhci0: USB revision 3.0
uhub0 at usb0 configuration 1 interface 0 "Generic xHCI root hub" rev 3.00/1.00 
addr 1
octxctl1 at simplebus0: DWC3 rev 0x250a
xhci1 at octxctl1, xHCI 1.0

Re: pf nat-to doesn't match a crafted packet

[David Gwynne]

How are you injecting the crafted packet into the stack?

On Tue, 29 Aug 2023, 01:14 ,  wrote:

> >Synopsis:  pf nat-to doesn't match a crafted packet
> >Category:  system
> >Environment:
> System  : OpenBSD 7.3
> Details : OpenBSD 7.3 (GENERIC.MP) #2080: Sat Mar 25 14:20:25
> MDT 2023
>  dera...@arm64.openbsd.org:
> /usr/src/sys/arch/arm64/compile/GENERIC.MP
>
> Architecture: OpenBSD.arm64
> Machine : arm64
> >Description:
> I was testing a seemingly valid Internet packet going out my
> gateway
> but the pf firewall doesn't match nat-to to this one for some reason.  I'm
> possibly overlooking something but every other packet exiting my gateway is
> nat'ed.  What causes this?  How can this be exploited?
>
> >How-To-Repeat:
> Here is the tcpdump from the host 1 hop behind the NAT router:
>
> 16:59:08.438082 192.168.177.13 > 49.12.42.182: icmp: host 7.198.187.211
> unreachable [icmp cksum ok] for 11.69.44.241.52699 > 7.198.187.211.55672:
> udp 51351 [tos 0x9c] (ttl 147, id 17124, len 51419, optlen=40 NOP RR{39}=
> RR{#106.155.117.54 233.26.79.111 129.127.249.242 60.117.146.16
> 179.39.29.224 213.65.49.78 0.16.45.109 252.168.188.0 123.108.138.224}) (ttl
> 64, id 65443, len 96)
>   : 4500 0060 ffa3  4001 ad81 c0a8 b10d  E..`@...
>   0010: 310c 2ab6 0301 55aa   4f9c c8db  1.*...U.O...
>   0020: 42e4  9311 c756 0b45 2cf1 07c6 bbd3  B..V.E,.
>   0030: 0107 2704 6a9b 7536 e91a 4f6f 817f f9f2  ..'.j.u6..Oo
>   0040: 3c75 9210 b327 1de0 d541 314e 0010 2d6d 0050: fca8 bc00 7b6c 8ae0 cddb d978    {l.x
>
> and here is the tcpdump on the pppoe interface:
>
> 16:59:08.440403 192.168.177.13 > 49.12.42.182: icmp: host 7.198.187.211
> unreacha
> ble [icmp cksum ok] (ttl 63, id 65443, len 96)
>
> Here is the relevant anchor rules I have:
>
>match out on $ext_if inet from  to any nat-to ($ext_if)
>
> and:
>
> table  const { 10/8, 172.16/12, 192.168/16 }
>
> Why did pf not translate this?  ... that's kinda kinky.
>
> >Fix:
> Not known.
>
>
> dmesg:
> OpenBSD 7.3 (GENERIC.MP) #2080: Sat Mar 25 14:20:25 MDT 2023
> dera...@arm64.openbsd.org:/usr/src/sys/arch/arm64/compile/GENERIC.MP
> real mem  = 8432840704 (8042MB)
> avail mem = 8139239424 (7762MB)
> random: good seed from bootblocks
> mainbus0 at root: ACPI
> psci0 at mainbus0: PSCI 1.1, SMCCC 1.2
> cpu0 at mainbus0 mpidr 0: ARM Cortex-A72 r0p3
> cpu0: 48KB 64b/line 3-way L1 PIPT I-cache, 32KB 64b/line 2-way L1 D-cache
> cpu0: 1024KB 64b/line 16-way L2 cache
> cpu0: CRC32,ASID16
> cpu1 at mainbus0 mpidr 1: ARM Cortex-A72 r0p3
> cpu1: 48KB 64b/line 3-way L1 PIPT I-cache, 32KB 64b/line 2-way L1 D-cache
> cpu1: 1024KB 64b/line 16-way L2 cache
> cpu1: CRC32,ASID16
> cpu2 at mainbus0 mpidr 2: ARM Cortex-A72 r0p3
> cpu2: 48KB 64b/line 3-way L1 PIPT I-cache, 32KB 64b/line 2-way L1 D-cache
> cpu2: 1024KB 64b/line 16-way L2 cache
> cpu2: CRC32,ASID16
> cpu3 at mainbus0 mpidr 3: ARM Cortex-A72 r0p3
> cpu3: 48KB 64b/line 3-way L1 PIPT I-cache, 32KB 64b/line 2-way L1 D-cache
> cpu3: 1024KB 64b/line 16-way L2 cache
> cpu3: CRC32,ASID16
> efi0 at mainbus0: UEFI 2.7
> efi0: https://github.com/pftf/RPi4 rev 0x1
> smbios0 at efi0: SMBIOS 3.3.0
> smbios0: vendor https://github.com/pftf/RPi4 version "UEFI Firmware
> v1.21" date 11/13/2020
> smbios0: Raspberry Pi Foundation Raspberry Pi 4 Model B
> apm0 at mainbus0
> ampintc0 at mainbus0 nirq 256, ncpu 4 ipi: 0, 1, 2: "interrupt-controller"
> agtimer0 at mainbus0: 54000 kHz
> acpi0 at mainbus0: ACPI 6.3
> acpi0: sleep states
> acpi0: tables DSDT FACP CSRT DBG2 GTDT IORT APIC PPTT BGRT
> acpi0: wakeup devices
> acpiiort0 at acpi0
> "BCM2849" at acpi0 not configured
> "BCM2835" at acpi0 not configured
> "BCM2854" at acpi0 not configured
> "ACPI0004" at acpi0 not configured
> xhci0 at acpi0 XHC0 addr 0x6/0x1000 irq 175, xHCI 1.0
> usb0 at xhci0: USB revision 3.0
> uhub0 at usb0 configuration 1 interface 0 "Generic xHCI root hub" rev
> 3.00/1.00 addr 1
> "ACPI0007" at acpi0 not configured
> "ACPI0007" at acpi0 not configured
> "ACPI0007" at acpi0 not configured
> "ACPI0007" at acpi0 not configured
> "ACPI0004" at acpi0 not configured
> "BCM2848" at acpi0 not configured
> "BCM2850" at acpi0 not configured
> "BCM2856" at acpi0 not configured
> "BCM2845" at acpi0 not configured
> "BCM2841" at acpi0 not configured
> "BCM2841" at acpi0 not configured
> "BCM2838" at acpi0 not configured
> "BCM2839" at acpi0 not configured
> "BCM2844" at acpi0 not configured
> pluart0 at acpi0 URT0 addr 0xfe201000/0x1000 irq 153
> "BCM2836" at acpi0 not configured
> "BCM2EA6" at acpi0 not configured
> "MSFT8000" at acpi0 not configured
> sdhc0 at acpi0 SDC1 addr 0xfe30/0x100 irq 158
> sdhc0: base clock frequency unknown
> "BCM2855" at acpi0 not configured
> bse0 at acpi0 ETH0 addr 0xfd58/0x1 irq 189: address
> dc:a6:32:cc:db:a7
> brgphy0 at bse0 phy 1: BCM54210E 10/100/1000baseT 

Re: find(1) with -delete ignores -maxdepth

[George Koehler]

On Mon, 28 Aug 2023 12:42:39 +0200
Marcus MERIGHI  wrote:

> My complete crontab(5) entry reads:
> 
> /usr/bin/find ~/ -maxdepth 1 -fstype local -name "*.core" -delete
> 
> Now I get error messages saying "Access Denied" for directories that
> find(1) should not access due to "-maxdepth 1". 

I can't produce such errors.  I observe that -maxdepth 1 and -delete
work well together.  There's no bug unless someone can provide more
specific instructions to produce the error.

Re: pf nat-to doesn't match a crafted packet

[Stuart Henderson]

On 2023/08/28 18:30, Peter J. Philipp wrote:
> Here is my icmp rulesets:
> 
> root@stern# grep icmp /etc/pf.conf

a partial pf.conf fragment is hardly ever enough to debug a ruleset
problem. if a packet doesn't match any rule then it hits the implicit
"pass flags any no state" rule 0.

Re: pf nat-to doesn't match a crafted packet

[Alexandr Nedvedicky]

Hello,

On Mon, Aug 28, 2023 at 06:30:55PM +0200, Peter J. Philipp wrote:
> 
> Hi Alexandr,
> 
> root@stern# tcpdump -v -n -i pppoe0 -c 1 icmp && pfctl -ss -v | grep icmp 
> tcpdump: listening on pppoe0, link-type PPP_ETHER
> 18:25:34.273661 192.168.177.13 > 49.12.42.182: icmp: host 7.198.187.211 
> unreachable [icmp cksum ok] (ttl 63, id 60642, len 96)
> root@stern# 
> 
> No state.  Though it's weird that this packet makes it out despite?

it depends on rules loaded to pf(4). how 'pfctl -sr' looks like
on your firewall?

if there is no rule which blocks outbound icmp then packet leaves
as is.

regards
sashan

Re: pf nat-to doesn't match a crafted packet

[Peter J. Philipp]

On Mon, Aug 28, 2023 at 06:18:41PM +0200, Alexandr Nedvedicky wrote:
> Hello,
> 
> On Mon, Aug 28, 2023 at 05:13:29PM +0200, p...@delphinusdns.org wrote:
> > >Synopsis:  pf nat-to doesn't match a crafted packet
> > >Category:  system
> > >Environment:
> > System  : OpenBSD 7.3
> > Details : OpenBSD 7.3 (GENERIC.MP) #2080: Sat Mar 25 14:20:25 MDT 
> > 2023
> >  
> > dera...@arm64.openbsd.org:/usr/src/sys/arch/arm64/compile/GENERIC.MP
> > 
> > Architecture: OpenBSD.arm64
> > Machine : arm64
> > >Description:
> > I was testing a seemingly valid Internet packet going out my gateway 
> > but the pf firewall doesn't match nat-to to this one for some reason.  I'm
> > possibly overlooking something but every other packet exiting my gateway is
> > nat'ed.  What causes this?  How can this be exploited?
> > 
> > >How-To-Repeat:
> > Here is the tcpdump from the host 1 hop behind the NAT router:
> > 
> > 16:59:08.438082 192.168.177.13 > 49.12.42.182: icmp: host 7.198.187.211 
> > unreachable [icmp cksum ok] for 11.69.44.241.52699 > 7.198.187.211.55672: 
> > udp 51351 [tos 0x9c] (ttl 147, id 17124, len 51419, optlen=40 NOP RR{39}= 
> > RR{#106.155.117.54 233.26.79.111 129.127.249.242 60.117.146.16 
> > 179.39.29.224 213.65.49.78 0.16.45.109 252.168.188.0 123.108.138.224}) (ttl 
> > 64, id 65443, len 96)
> >   : 4500 0060 ffa3  4001 ad81 c0a8 b10d  E..`@...
> >   0010: 310c 2ab6 0301 55aa   4f9c c8db  1.*...U.O...
> >   0020: 42e4  9311 c756 0b45 2cf1 07c6 bbd3  B..V.E,.
> >   0030: 0107 2704 6a9b 7536 e91a 4f6f 817f f9f2  ..'.j.u6..Oo
> >   0040: 3c75 9210 b327 1de0 d541 314e 0010 2d6d   >   0050: fca8 bc00 7b6c 8ae0 cddb d978    {l.x
> > 
> > and here is the tcpdump on the pppoe interface:
> > 
> 
> can you check there is a state in pf(4) matching ICMP dest unreachable
> packet?
> 
> in order to handle icmp unreachable message there must be matching
> state in pf(4).
> 
> refer to pf_test_state_icmp() where translation of ICMP error messages
> happens.
> 
> hope it helps
> regards
> sashan
> 

Hi Alexandr,

root@stern# tcpdump -v -n -i pppoe0 -c 1 icmp && pfctl -ss -v | grep icmp 
tcpdump: listening on pppoe0, link-type PPP_ETHER
18:25:34.273661 192.168.177.13 > 49.12.42.182: icmp: host 7.198.187.211 
unreachable [icmp cksum ok] (ttl 63, id 60642, len 96)
root@stern# 

No state.  Though it's weird that this packet makes it out despite?

Is it supposed to work this way?  As far as I understand it the packet made 
it to the link layer on its way out and pf doesn't stop it there any more.

Here is my icmp rulesets:

root@stern# grep icmp /etc/pf.conf
pass in on pppoe0 inet proto icmp all icmp-type 8 code 0  keep state
pass out on $port1 inet6 proto icmp6 all keep state
pass in on $port1 inet proto icmp from 192.168.177.10 to any keep state
pass in on $port1 inet proto icmp from any to any icmp-type 8 code 0 
keep state
pass in on $port1 inet proto icmp from any to any icmp-type 3 code 1 
keep state
pass out log (all) on vlan4 inet proto icmp all
pass in log (all) on vlan4 inet proto icmp all

port1 is the ethernet interface facing my LAN (bse0).

Best Regards,
-peter

-- 
Over thirty years experience on Unix-like Operating Systems starting with QNX.

Re: pf nat-to doesn't match a crafted packet

[Alexandr Nedvedicky]

Hello,

On Mon, Aug 28, 2023 at 05:13:29PM +0200, p...@delphinusdns.org wrote:
> >Synopsis:pf nat-to doesn't match a crafted packet
> >Category:system
> >Environment:
>   System  : OpenBSD 7.3
>   Details : OpenBSD 7.3 (GENERIC.MP) #2080: Sat Mar 25 14:20:25 MDT 
> 2023
>
> dera...@arm64.openbsd.org:/usr/src/sys/arch/arm64/compile/GENERIC.MP
> 
>   Architecture: OpenBSD.arm64
>   Machine : arm64
> >Description:
>   I was testing a seemingly valid Internet packet going out my gateway 
> but the pf firewall doesn't match nat-to to this one for some reason.  I'm
> possibly overlooking something but every other packet exiting my gateway is
> nat'ed.  What causes this?  How can this be exploited?
> 
> >How-To-Repeat:
> Here is the tcpdump from the host 1 hop behind the NAT router:
> 
> 16:59:08.438082 192.168.177.13 > 49.12.42.182: icmp: host 7.198.187.211 
> unreachable [icmp cksum ok] for 11.69.44.241.52699 > 7.198.187.211.55672: udp 
> 51351 [tos 0x9c] (ttl 147, id 17124, len 51419, optlen=40 NOP RR{39}= 
> RR{#106.155.117.54 233.26.79.111 129.127.249.242 60.117.146.16 179.39.29.224 
> 213.65.49.78 0.16.45.109 252.168.188.0 123.108.138.224}) (ttl 64, id 65443, 
> len 96)
>   : 4500 0060 ffa3  4001 ad81 c0a8 b10d  E..`@...
>   0010: 310c 2ab6 0301 55aa   4f9c c8db  1.*...U.O...
>   0020: 42e4  9311 c756 0b45 2cf1 07c6 bbd3  B..V.E,.
>   0030: 0107 2704 6a9b 7536 e91a 4f6f 817f f9f2  ..'.j.u6..Oo
>   0040: 3c75 9210 b327 1de0 d541 314e 0010 2d6d 0050: fca8 bc00 7b6c 8ae0 cddb d978    {l.x
> 
> and here is the tcpdump on the pppoe interface:
> 

can you check there is a state in pf(4) matching ICMP dest unreachable
packet?

in order to handle icmp unreachable message there must be matching
state in pf(4).

refer to pf_test_state_icmp() where translation of ICMP error messages
happens.

hope it helps
regards
sashan

pf nat-to doesn't match a crafted packet

[pjp]

>Synopsis:  pf nat-to doesn't match a crafted packet
>Category:  system
>Environment:
System  : OpenBSD 7.3
Details : OpenBSD 7.3 (GENERIC.MP) #2080: Sat Mar 25 14:20:25 MDT 
2023
 
dera...@arm64.openbsd.org:/usr/src/sys/arch/arm64/compile/GENERIC.MP

Architecture: OpenBSD.arm64
Machine : arm64
>Description:
I was testing a seemingly valid Internet packet going out my gateway 
but the pf firewall doesn't match nat-to to this one for some reason.  I'm
possibly overlooking something but every other packet exiting my gateway is
nat'ed.  What causes this?  How can this be exploited?

>How-To-Repeat:
Here is the tcpdump from the host 1 hop behind the NAT router:

16:59:08.438082 192.168.177.13 > 49.12.42.182: icmp: host 7.198.187.211 
unreachable [icmp cksum ok] for 11.69.44.241.52699 > 7.198.187.211.55672: udp 
51351 [tos 0x9c] (ttl 147, id 17124, len 51419, optlen=40 NOP RR{39}= 
RR{#106.155.117.54 233.26.79.111 129.127.249.242 60.117.146.16 179.39.29.224 
213.65.49.78 0.16.45.109 252.168.188.0 123.108.138.224}) (ttl 64, id 65443, len 
96)
  : 4500 0060 ffa3  4001 ad81 c0a8 b10d  E..`@...
  0010: 310c 2ab6 0301 55aa   4f9c c8db  1.*...U.O...
  0020: 42e4  9311 c756 0b45 2cf1 07c6 bbd3  B..V.E,.
  0030: 0107 2704 6a9b 7536 e91a 4f6f 817f f9f2  ..'.j.u6..Oo
  0040: 3c75 9210 b327 1de0 d541 314e 0010 2d6d   49.12.42.182: icmp: host 7.198.187.211 unreacha
ble [icmp cksum ok] (ttl 63, id 65443, len 96)

Here is the relevant anchor rules I have:

   match out on $ext_if inet from  to any nat-to ($ext_if)

and:

table  const { 10/8, 172.16/12, 192.168/16 }

Why did pf not translate this?  ... that's kinda kinky.

>Fix:
Not known.


dmesg:
OpenBSD 7.3 (GENERIC.MP) #2080: Sat Mar 25 14:20:25 MDT 2023
dera...@arm64.openbsd.org:/usr/src/sys/arch/arm64/compile/GENERIC.MP
real mem  = 8432840704 (8042MB)
avail mem = 8139239424 (7762MB)
random: good seed from bootblocks
mainbus0 at root: ACPI
psci0 at mainbus0: PSCI 1.1, SMCCC 1.2
cpu0 at mainbus0 mpidr 0: ARM Cortex-A72 r0p3
cpu0: 48KB 64b/line 3-way L1 PIPT I-cache, 32KB 64b/line 2-way L1 D-cache
cpu0: 1024KB 64b/line 16-way L2 cache
cpu0: CRC32,ASID16
cpu1 at mainbus0 mpidr 1: ARM Cortex-A72 r0p3
cpu1: 48KB 64b/line 3-way L1 PIPT I-cache, 32KB 64b/line 2-way L1 D-cache
cpu1: 1024KB 64b/line 16-way L2 cache
cpu1: CRC32,ASID16
cpu2 at mainbus0 mpidr 2: ARM Cortex-A72 r0p3
cpu2: 48KB 64b/line 3-way L1 PIPT I-cache, 32KB 64b/line 2-way L1 D-cache
cpu2: 1024KB 64b/line 16-way L2 cache
cpu2: CRC32,ASID16
cpu3 at mainbus0 mpidr 3: ARM Cortex-A72 r0p3
cpu3: 48KB 64b/line 3-way L1 PIPT I-cache, 32KB 64b/line 2-way L1 D-cache
cpu3: 1024KB 64b/line 16-way L2 cache
cpu3: CRC32,ASID16
efi0 at mainbus0: UEFI 2.7
efi0: https://github.com/pftf/RPi4 rev 0x1
smbios0 at efi0: SMBIOS 3.3.0
smbios0: vendor https://github.com/pftf/RPi4 version "UEFI Firmware v1.21" date 
11/13/2020
smbios0: Raspberry Pi Foundation Raspberry Pi 4 Model B
apm0 at mainbus0
ampintc0 at mainbus0 nirq 256, ncpu 4 ipi: 0, 1, 2: "interrupt-controller"
agtimer0 at mainbus0: 54000 kHz
acpi0 at mainbus0: ACPI 6.3
acpi0: sleep states
acpi0: tables DSDT FACP CSRT DBG2 GTDT IORT APIC PPTT BGRT
acpi0: wakeup devices
acpiiort0 at acpi0
"BCM2849" at acpi0 not configured
"BCM2835" at acpi0 not configured
"BCM2854" at acpi0 not configured
"ACPI0004" at acpi0 not configured
xhci0 at acpi0 XHC0 addr 0x6/0x1000 irq 175, xHCI 1.0
usb0 at xhci0: USB revision 3.0
uhub0 at usb0 configuration 1 interface 0 "Generic xHCI root hub" rev 3.00/1.00 
addr 1
"ACPI0007" at acpi0 not configured
"ACPI0007" at acpi0 not configured
"ACPI0007" at acpi0 not configured
"ACPI0007" at acpi0 not configured
"ACPI0004" at acpi0 not configured
"BCM2848" at acpi0 not configured
"BCM2850" at acpi0 not configured
"BCM2856" at acpi0 not configured
"BCM2845" at acpi0 not configured
"BCM2841" at acpi0 not configured
"BCM2841" at acpi0 not configured
"BCM2838" at acpi0 not configured
"BCM2839" at acpi0 not configured
"BCM2844" at acpi0 not configured
pluart0 at acpi0 URT0 addr 0xfe201000/0x1000 irq 153
"BCM2836" at acpi0 not configured
"BCM2EA6" at acpi0 not configured
"MSFT8000" at acpi0 not configured
sdhc0 at acpi0 SDC1 addr 0xfe30/0x100 irq 158
sdhc0: base clock frequency unknown
"BCM2855" at acpi0 not configured
bse0 at acpi0 ETH0 addr 0xfd58/0x1 irq 189: address dc:a6:32:cc:db:a7
brgphy0 at bse0 phy 1: BCM54210E 10/100/1000baseT PHY, rev. 2
"PNP0C06" at acpi0 not configured
acpitz0 at acpi0: critical temperature is 90 degC
simplefb0 at mainbus0: 640x480, 32bpp
wsdisplay0 at simplefb0 mux 1: console (std, vt100 emulation)
wsdisplay0: screen 1-5 added (std, vt100 emulation)
uhub1 at uhub0 port 1 configuration 1 interface 0 "VIA Labs USB2.0 Hub" rev 
2.10/4.21 addr 2
uhidev0 at uhub1 port 4 configuration 1 interface 0 "APC Back-UPS ES 700G 
FW:871.O4 .I USB FW:O4" rev 1.10/1.06 addr 3

find(1) with -delete ignores -maxdepth

[Marcus MERIGHI]

Hallo, 

a recent post on misc@ mentioned the following crontab(5) command to
remove core files:

@reboot /usr/bin/find ~ -fstype local -name '*.core' -delete
( https://marc.info/?l=openbsd-misc=169303728425504 )

I wondered why I did not use the "-delete" primary but the "-print0 |
xargs -0" pipeline, back when I added the cron entry. 
Thus I change the crontab entry to use "-delete".

I have another primary in my parameters to find: "-maxdepth 1".

My complete crontab(5) entry reads:

/usr/bin/find ~/ -maxdepth 1 -fstype local -name "*.core" -delete

Now I get error messages saying "Access Denied" for directories that
find(1) should not access due to "-maxdepth 1". 

(Please do not ask why the user has directories within her/his $HOME
that she/he cannot access. :-)

Does the "-delete" primary lead to ignoring "-maxdepth 1"?

Thanks for your time, Marcus