Re: Potential bgpd.conf parsing bug?

[Jordan Geoghegan]

Hi Theo,

I appreciate you taking the time to respond, thanks for the insight. 
That solves that mystery.


I just wanted to take the opportunity to say Happy Birthday Theo, and 
thank you for making the greatest OS and software ecosystem around.
With no hyperbole, OpenBSD has changed my life - it facilitated me 
overcoming some serious adversity when I first went out on my own and 
has given me the ability to earn a respectable living, and for that I 
thank you.


Regards,

Jordan

On 5/19/23 06:33, Theo de Raadt wrote:

Many of our daemons (and a few non-daemon tools) contain a a particular
parse.y parser for their domain-specific-languages.

This is how variables work in that parser.  This is kind of like a
historical mistake that would be hard to repair, because everytime we
looked into changing the way it works (by moving the expansion to a different
stage) it would break other user patterns.

Jordan Geoghegan  wrote:


Hello,

I was playing around with my bgpd.conf and I noticed some interesting
behaviour with macros.

A quick example:

# This parses fine:
IP1="192.0.2.1"
MYIP=$IP1

# This throws a syntax error:
ASN1="65001"
MYASN=$ASN1

# This also parses fine:
ASN1="65001a"
MYASN=$ASN1

In short, it appears that having a numerical value for macro2=$macro1
breaks something in the config parser, or at the very least the
behaviour is undocumented.

Any insight or advice would be much appreciated.

Regards,

Jordan

Potential bgpd.conf parsing bug?

[Jordan Geoghegan]

Hello,

I was playing around with my bgpd.conf and I noticed some interesting 
behaviour with macros.


A quick example:

# This parses fine:
IP1="192.0.2.1"
MYIP=$IP1

# This throws a syntax error:
ASN1="65001"
MYASN=$ASN1

# This also parses fine:
ASN1="65001a"
MYASN=$ASN1

In short, it appears that having a numerical value for macro2=$macro1 
breaks something in the config parser, or at the very least the 
behaviour is undocumented.


Any insight or advice would be much appreciated.

Regards,

Jordan

Re: passing environment variables to daemons in rc.d scripts

[Jordan Geoghegan]

Hi Antoine, Marcus and Stuart,

Thank you all for your help - setenv via login class was exactly what I 
was looking for!


On 4/20/23 23:44, Stuart Henderson wrote:

On 2023-04-21, Antoine Jacoutot  wrote:

Hi.

You can pas environment variables by creating a login class matching the name 
of your rc.d script in login.conf and adding setenv to that class.

Or create a new file /etc/login.conf.d/(script_name). If you're writing
a port, create pkg/(script_name).login and it will be added to the plist
automatically when you "make update-list".

Thanks for that, worked perfectly!




The problem I'm facing is that it seems that Airflow looks for various 
environment variables such as $HOME, $AIRFLOW_HOME, $AIRFLOW_CONFIG etc and I'm 
seeing no obvious way to pass those requisite environment variables to Airflow 
from my rc.d script. Without these variables set, Airflow annoyingly just looks 
in /dev/null for everything and fails to function.

That seems a strange dir to use by default. Did you perhaps set the
_airflowd user's home dir to /dev/null? If so, try /var/airflow instead.




The users home folder was indeed set to /var/airflow. Turns out the 
/dev/null thing was a red herring - I appear to have found a bug in 
Airflow. When starting the Airflow scheduler with '-D' to daemonize, the 
scheduler program just silently hangs. I got sent down a couple rabbit 
holes because the scheduler continued to send heartbeats yet failed to 
parse dags or run jobs. I didn't fully clue in until I compared the 
process lists between the scheduler when daemonized vs running it in the 
foreground. When run in the forground, all sorts of LocalExecutor and 
other tasks were spawned, unlike when daemonized. It was then made super 
obvious when I realized when daemonized it was leaving zombie processes 
behind after stopping with rcctl/SIGTERM. Curiously, the airflow 
webserver seems to daemonize just fine and I've had no such issues with it.


Long story short, after removing '-D' flag from my rc.d daemon 
arguments, everything worked and I didn't even end up actually needing 
to set any environment variables.


With that said, I'll put together a bug report for the Airflow folks.

Thank you muchly for the pointers.

Regards,

Jordan

passing environment variables to daemons in rc.d scripts

[Jordan Geoghegan]

Hello,

tl;dr: Is there any way to pass an environment variable to a daemon 
started with rc.d?


A bit of context for those interested:

I'm trying to run Apache Airflow from an rc.d script so I can make use 
of rcctl and other niceties. My rc.d script is included below.


The problem I'm facing is that it seems that Airflow looks for various 
environment variables such as $HOME, $AIRFLOW_HOME, $AIRFLOW_CONFIG etc 
and I'm seeing no obvious way to pass those requisite environment 
variables to Airflow from my rc.d script. Without these variables set, 
Airflow annoyingly just looks in /dev/null for everything and fails to 
function.


I'm probably missing something obvious, but hoping the fine folks here 
can point me in the right direction.


Regards,

Jordan


# Airflow scheduler rc.d script:

#!/bin/ksh
#

daemon="/usr/local/bin/airflow scheduler -D"
daemon_flags="-l - --stderr - --stdout -"
daemon_user="_airflowd"
daemon_logger="daemon.info"
daemon_timeout="60"

. /etc/rc.d/rc.subr

pexp=".*python.* ${daemon} ${daemon_flags}"
rc_reload=NO

rc_pre() {
    rm -f /var/airflow/airflow/airflow-scheduler.pid
}

rc_cmd $1


# Airflow webserver r rc.d script:

#!/bin/ksh
#

daemon="/usr/local/bin/airflow webserver -D -E -"
daemon_flags="-p 8080 -l - --stderr - --stdout -"
daemon_user="_airflowd"
daemon_logger="daemon.info"

. /etc/rc.d/rc.subr

pexp=".*python.* ${daemon} ${daemon_flags}"
rc_reload=NO

rc_pre() {
    rm -f /var/airflow/airflow/airflow-webserver.pid \
    /var/airflow/airflow/airflow-webserver-monitor.pid
}

rc_cmd $1

Re: Using gzip-static with httpd location

[Jordan Geoghegan]

On 3/9/23 17:31, Joel Carnat wrote:

Hi,

I just tried applying gzip compression on a simple test web site using 
httpd and the gzip-static option ; using OpenBSD 7.2/amd64.


As I understood the man page, gzip-static is supposed to be used 
inside the server block ; like listen, errdocs or tls. But doing so 
does not seem to enable gzip compression for files defined in a 
location block.


What fails:
server "default" {
  listen on 127.0.0.1 port 80
  gzip-static
  block drop
  location "/.well-known/acme-challenge/*" {
    root "/acme"
    request strip 2
    pass
  }
  location "/www/*" {
    root "/test"
    request strip 1
    pass
  }
}

What works:
server "default" {
  listen on 127.0.0.1 port 80
  block drop
  location "/.well-known/acme-challenge/*" {
    root "/acme"
    request strip 2
    pass
  }
  location "/www/*" {
    gzip-static
    root "/test"
    request strip 1
    pass
  }
}

As you may see, what works is using gzip-static inside a location 
block and not outside. I've tested is using Firefox, curl and 
https://gtmetrix.com. All confirm gzip-static must be inside the 
location block to provide compressed resources.


Here's an example of the curl command I used:
# curl -I --compressed http://localhost:80/www/index.html
HTTP/1.1 200 OK
Connection: keep-alive
Content-Encoding: gzip
Content-Length: 1083
Content-Type: text/html
Date: Fri, 10 Mar 2023 01:27:53 GMT
Last-Modified: Fri, 10 Mar 2023 00:53:26 GMT
Server: OpenBSD httpd

Is this an expected behaviour?

Regards,
Joel C.



Can confirm - I recently stumbled over this confusing behaviour as well. 
Curious if this is a bug or a man page issue.


Regards,

Jordan

Re: some simple way to serve videos?

[Jordan Geoghegan]

On 10/3/22 05:00, rsyk...@disroot.org wrote:

Hello,


until now I have www-served (httpd) my photos using, as it seems to me,
a very simple way: into a directory with photos I copied a file called
gallery.html taken from

https://github.com/gfwilliams/ThinGallery
.

This created a browsable gallery of photos (using a web browser and
over the internet), simple enough for my mom to orient in and use.
Is there any similar way so that I could serve also video files?

Thanks for any comments / recommendations.

(I want something that lives on my machine. I know there are some 'big'
frameworks, perhaps MediaGoblin, Serviio; also I could perhaps run a docker
with minnich under a virtual machine runing linux. But is there something
easy and available on OpenBSD?)


Best regards,
Ruda



I've successfully run Serviio on OpenBSD in the past and it worked 
great. The only problem I encountered was an FFmpeg issue where subtitle 
burn-in options didn't work.


Regards,

Jordan

Re: calling all PFsync users for experience, gotchas, feedback, tips and tricks

[Jordan Geoghegan]

On 5/11/22 12:32, Tom Smyth wrote:

Hello Folks,

We are updating some course material for an upcoming PF firewall course,
and I would like to put a call out to those who use PFsync in a
redundant firewall cluster
about your user experience, have you come across any edge cases?
have you any tips or tricks about PFSync.
have you come across any edge cases / minor misconfigurations /
suboptimal configurations that caused problems, were there some tweaks
you had to make to make your system scale ?

it is likely that people who are running PFSync have  more complicated
firewall configs.

and I would like to see what tuning other people have done in the field.

I would appreciate any feedback or problem descriptions  (with our
without solutions)

what is the largest throughput firewall you deployed with PFSync?  how
was your experience
of running with PFsync vs without PFsync  on your firewall.

Thanks again,




I've run pfsync + CARP for a number of years now. One interesting 
"gotcha" I discovered when building an IPv6-only test network was that 
pfsync does not work in an IPv6-only environment. I tried both unicast 
and multicast configurations to no avail. When pfsync has a parent 
interface that only has an IPv6 address assigned (ie no IPv4 at all), no 
pfsync traffic transits the interface. Just thought I'd share this 
little tidbit since you were looking for edge cases and gotchas and 
since IPv6 support (or lack thereof) is not mentioned in the manpage.


Regards,

Jordan

HP Probook audio only playing on left speaker

[Jordan Geoghegan]

Hello,

I have an HP Probook (dmesg below) that is only playing audio from the 
left headphone speaker. No matter how I try I cannot get any audio to 
play over the right side speaker. I have very little experience 
debugging audio issues on OpenBSD as everything has largely "just 
worked" for me over the years in that regard. Any insight or advice 
would be greatly appreciated!


Here is the output from mixerctl:

inputs.dac-0:1_mute=off
inputs.dac-0:1=207,207
inputs.dac-2:3_mute=off
inputs.dac-2:3=207,207
inputs.beep=108
record.adc-2:3_source=mic
record.adc-2:3_mute=off
record.adc-2:3=240,240
record.adc-0:1_source=mic2
record.adc-0:1_mute=off
record.adc-0:1=240,240
outputs.spkr_source=dac-2:3
outputs.spkr_eapd=on
inputs.mic=85,85
outputs.mic_dir=input-vr80
inputs.mic2=85,85
outputs.hp_source=dac-0:1
inputs.hp=85,85
outputs.hp_dir=output
outputs.hp_boost=off
outputs.hp_eapd=on
outputs.mic_sense=unplugged
outputs.hp_sense=unplugged
outputs.spkr_muters=hp
outputs.master=207,207
outputs.master.mute=off
outputs.master.slaves=dac-0:1,dac-2:3
record.volume=255,255
record.volume.mute=off
record.volume.slaves=adc-2:3,adc-0:1
record.enable=sysctl

dmesg:

OpenBSD 7.0 (GENERIC.MP) #2: Wed Nov 24 00:54:11 MST 2021
r...@syspatch-70-amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 8409149440 (8019MB)
avail mem = 8138280960 (7761MB)
random: good seed from bootblocks
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 3.1 @ 0xcf764000 (33 entries)
bios0: vendor HP version "Q83 Ver. 01.07.00" date 04/17/2019
bios0: HP HP ProBook 640 G4
acpi0 at bios0: ACPI 5.0
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP SSDT SSDT RTMA SSDT SSDT UEFI SSDT TPM2 SSDT 
MSDM SLIC WSMT HPET APIC MCFG SSDT SSDT SSDT SSDT SSDT SSDT SSDT SSDT 
SSDT SSDT DBGP DBG2 DMAR NHLT SSDT SSDT SSDT ASF! FPDT BGRT SSDT
acpi0: wakeup devices GLAN(S4) XHC_(S3) XDCI(S4) HDAS(S4) RP01(S4) 
PXSX(S4) RP02(S4) PXSX(S4) RP03(S4) PXSX(S4) RP04(S4) PXSX(S4) RP05(S4) 
PXSX(S4) RP06(S4) PXSX(S4) [...]

acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpihpet0 at acpi0: 2399 Hz
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz, 1496.52 MHz, 06-8e-0a
cpu0: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SGX,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PT,SRBDS_CTRL,MD_CLEAR,TSXFA,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES,MELTDOWN

cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 24MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4.1.1.1, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz, 1496.52 MHz, 06-8e-0a
cpu1: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SGX,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PT,SRBDS_CTRL,MD_CLEAR,TSXFA,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES,MELTDOWN

cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz, 1496.52 MHz, 06-8e-0a
cpu2: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SGX,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PT,SRBDS_CTRL,MD_CLEAR,TSXFA,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES,MELTDOWN

cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 6 (application processor)
cpu3: Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz, 1496.52 MHz, 06-8e-0a
cpu3: 

Re: relayd unable to get local issuer certificate

[Jordan Geoghegan]

On 12/8/21 00:10, Anthony J. Bentley wrote:

Jordan Geoghegan writes:

I generated a TLS cert with acme-client and  tested and confirmed it
worked with httpd.

Do curl/wget/ftp behave the same with httpd? If so that would imply
the problem is with the certificate.


I then configured relayd to perform TLS acceleration
by following examples in the man pages. Everything works great when
tested from a web browser. However, when I try to fetch a file via curl,
wget or ftp etc, I get a cert validation error: (ip and domain removed
for privacy)

Did you generate a full chain certificate with acme-client?




Okay, mystery solved - I had my morning coffee and then it was obvious 
after your hint re fullchain certificate.


relayd wasn't loading the pem file, it was loading the crt file, which 
must be fullchain it seems. A tidbit I discovered on solene@'s blog also 
confirmed this:
"For convenience, you will want to replace the path for the full chain 
certificate to have `hostname.crt` instead of `hostname.fullchain.pem` 
to match relayd expectations. "


Everything is working as expected now that I've made the changes.

Sorry for the noise.

Regards,

Jordan

Re: relayd unable to get local issuer certificate

[Jordan Geoghegan]

On 12/8/21 00:10, Anthony J. Bentley wrote:

Jordan Geoghegan writes:

I generated a TLS cert with acme-client and  tested and confirmed it
worked with httpd.

Do curl/wget/ftp behave the same with httpd? If so that would imply
the problem is with the certificate.


I then configured relayd to perform TLS acceleration
by following examples in the man pages. Everything works great when
tested from a web browser. However, when I try to fetch a file via curl,
wget or ftp etc, I get a cert validation error: (ip and domain removed
for privacy)

Did you generate a full chain certificate with acme-client?


Hi Anthony,

Thanks for your response!

curl/wget/ftp work perfectly with httpd, that's what has me confused. As 
soon as that cert gets loaded into relayd, curl/ftp/wget complain about 
certificates, yet weirdly web browsers are perfectly happy.


Yes, I do have a full chain certificate generated - I have *.key, *.crt 
and *.pem generated.


In the past, I've generated hundreds of certificates just like this one 
to use in my httpd hosting cluster - I'm very confused as to why relayd 
is puking with the certificate.


I'm happy to provide any further information that is needed, or even 
provide remote access to the machine if desired. This is just a throw 
away proof of concept test rig, so happy to assist in any way I can.


Regards,

Jordan

relayd unable to get local issuer certificate

[Jordan Geoghegan]

Hello,

I decided to try using relayd as a TLS accelerator and have run into an 
issue - as usual, I'm sure I'm missing something obvious.


I generated a TLS cert with acme-client and  tested and confirmed it 
worked with httpd. I then configured relayd to perform TLS acceleration 
by following examples in the man pages. Everything works great when 
tested from a web browser. However, when I try to fetch a file via curl, 
wget or ftp etc, I get a cert validation error: (ip and domain removed 
for privacy)


...
$ curl -iv https://www.example.com:8088/test.txt
*   Trying 192.0.1.189:8088...
* Connected to www.example.com (192.0.1.189) port 8088 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Unknown (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* SSL certificate problem: unable to get local issuer certificate
* Closing connection 0
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
...

My relayd config looks like this:
...
http protocol https {
    match request header append "X-Forwarded-For" value "$REMOTE_ADDR"
    match request header append "X-Forwarded-By" \
    value "$SERVER_ADDR:$SERVER_PORT"
    match request header set "Connection" value "close"

    # Various TCP options
    tcp { sack, backlog 128 }

    tls { no tlsv1.0, ciphers HIGH, keypair example.com }
#   tls no session tickets
}

relay test {
    # Run as a SSL/TLS accelerator
    listen on $ext_addr port 8088 tls
    protocol https

    # Forward to hosts in the webhosts table using a src/dst hash
    forward to  port 80 mode loadbalance \
    check icmp
}
...

I'm not sure what the issue is here, this is my first foray into relayd 
TLS acceleration, so I was hoping to get a quick sanity check here to 
make sure I'm not missing something obvious. I've certainly put some 
legwork into figuring this out, I spent all day futzing with this to no 
avail. Everything here is bog standard, so I'm curious what I'm doing wrong.


As usual, any insight or advice would be greatly appreciated.

Regards,

Jordan

Re: nvme boot

[Jordan Geoghegan]

On 10/15/21 8:05 AM, Jan Stary wrote:
> Does any of the OpenSBD-supported platforms boot off nvme storage?
> So far, I have been able to use nvme storage as a disk,
> but not boot from it; but my HW is far from recent.
>
>   Jan
>

Hi Jan,

NVME boot will require that your motherboard / bios support it. It sounds like 
you're machine does not support it.

I know OpenBSD supports NVME boot as I'm writing to you on an AMD Ryzen machine 
with B350 motherboard/chipset booting off of a 2TB NVME drive. Most modern 
laptops these days also boot off of NVME etc.

Regards,

Jordan

Re: pkg_info -m: libraries and dependencies marked as manually installed

[Jordan Geoghegan]

On 9/19/21 3:21 PM, Stuart Henderson wrote:
> On 2021-09-19, Jordan Geoghegan  wrote:
>> Hello,
>>
>> I recently decided to clean up my desktop machine of unused packages etc. 
>> I've been running release/stable on this machine since around the OpenBSD 
>> 6.2 era. The machine has been upgraded over the years all the way to 6.9. 
>> I'm not sure that it's relevant, but I've regularly run sysclean in an 
>> effort to keep my install clean and fresh.
>>
>> When running "pkg_info -m", within the output list I am shown a number of 
>> random dependencies which I did not manually install. Is there a way to 
>> remove the "manually installed" tag from these library/dependency packages 
>> to allow them to potentially be cleaned up by "pkg_delete -a"?
> Yes, with pkg_add(1):
>
> "-aa  Force already installed packages to be tagged as
>   installed automatically."
>
>

Thanks Stuart, you're a saint - that certainly did the trick!

Regards,

Jordan

pkg_info -m: libraries and dependencies marked as manually installed

[Jordan Geoghegan]

Hello,

I recently decided to clean up my desktop machine of unused packages etc. I've 
been running release/stable on this machine since around the OpenBSD 6.2 era. 
The machine has been upgraded over the years all the way to 6.9. I'm not sure 
that it's relevant, but I've regularly run sysclean in an effort to keep my 
install clean and fresh.

When running "pkg_info -m", within the output list I am shown a number of 
random dependencies which I did not manually install. Is there a way to remove 
the "manually installed" tag from these library/dependency packages to allow 
them to potentially be cleaned up by "pkg_delete -a"?


$ pkg_info -m
ImageMagick-6.9.12.2 image processing tools
adwaita-icon-theme-3.38.0 base icon theme for GNOME
aggregate-1.6p1 optimise a list of route prefixes
aggregate6-1.0.12p1 optimise a list of IPv4/IPv6 prefixes
amdgpu-firmware-20201218 firmware binary images for amdgpu(4) driver
aria2-1.35.0p0  lightweight multi-protocol & multi-source download utility
at-spi2-atk-2.38.0  atk-bridge for at-spi2
at-spi2-core-2.38.0 service interface for assistive technologies
atk-2.36.0  accessibility toolkit used by gtk+
avahi-0.8p0 framework for Multicast DNS Service Discovery
cairo-1.16.0    vector graphics library
cloc-1.86   count lines of code
coreutils-8.32p0    file, shell and text manipulation utilities
cowsay-3.04 speaking ascii cow
cups-libs-2.3.3.2   CUPS libraries and headers
dbus-glib-0.112v0   glib bindings for dbus message system
dconf-0.40.0    configuration backend system
desktop-file-utils-0.26 utilities for dot.desktop entries
dolphin-5.0.0.20190621p1 Nintendo GameCube and Wii emulator with GUI
dvd+rw-tools-7.1p1  mastering tools for DVD+RW/+R/-R/-RW
e2fsprogs-1.42.12p5 utilities to manipulate ext2 filesystems
evince-3.38.2-light GNOME document viewer
fdupes-2.1.2    identify or delete duplicate files
firefox-esr-78.14.0 Firefox web browser, Extended Support Release
freerdp-2.3.2   client for Microsoft RDP (remote desktop protocol)
gcr-3.38.1  library for bits of crypto UI and parsing
gdbm-1.19   GNU dbm
gdk-pixbuf-2.42.4   image data transformation library
gegl04-0.4.30p0 graph based image processing framework (API version 0.4)
geoclue-0.12.99p9   modular geoinformation service on top of D-Bus
gettext-runtime-0.21p1 GNU gettext runtime libraries and programs
ggrep-3.6   GNU versions of grep pattern matching utilities
gimp-2.10.24    GNU Image Manipulation Program
git-2.31.1  distributed version control system
glib2-2.66.8    general-purpose utility library
glib2-networking-2.66.0 network-related gio modules for GLib
gnome-icon-theme-3.12.0p5 base icon theme for GNOME
gnome-icon-theme-symbolic-3.12.0p3 base icon theme extension for special UI 
contexts
gnupg-2.2.23p2  GNU privacy guard - a free PGP replacement
gnutls-3.6.16   GNU Transport Layer Security library
go-1.16.2   Go programming language
gsettings-desktop-schemas-3.38.0 collection of shared GSettings schemas
gtk+2-2.24.33p0 multi-platform graphical toolkit
gtk+3-3.24.28p1 multi-platform graphical toolkit
gtk-update-icon-cache-3.24.28 gtk+ icon theme caching utility
gvfs-1.46.2 GNOME Virtual File System
gxemul-0.6.3.1  ARM/M88k/MIPS/PowerPC/SuperH machines emulator
harfbuzz-2.8.0  text shaping library
harfbuzz-icu-2.8.0  ICU support for libharfbuzz
httrack-3.48.21p2   offline browser
iwm-firmware-20191022p1 firmware binary images for iwm(4) driver
keepassxc-2.6.4 management tool for password and sensitive data
ksh93-20120801p1    AT Korn Shell
libassuan-2.5.5 IPC library used by GnuPG and gpgme
libcroco-0.6.13p0   generic CSS parsing library for GNOME project
libexif-0.6.22  extract digital camera info tags from JPEG images
libgcrypt-1.9.4 crypto library based on code used in GnuPG
libgpg-error-1.42   error codes for GnuPG related software
libidn-1.36 internationalized string handling
libksba-1.4.0   X.509 library
libmypaint-1.6.1    library for making brushstrokes
libnotify-0.7.9 send desktop notifications to a notification daemon
libproxy-0.4.17 library handling all the details of proxy configuration
libpsl-0.21.1   public suffix list library
libreoffice-7.0.5.2v0 multi-platform productivity suite
librsvg-2.50.3  SAX-based render library for SVG files
libsecret-0.20.4    library for storing and retrieving passwords and secrets
libsoup-2.72.0  HTTP client/server library for GNOME
mawk-1.3.4.20200120p0 fast POSIX-compliant awk
mupdf-1.18.0    graphic library, pdf parser, viewer and utilities
mupen64plus-2.5.9   n64 emulator (default plugins)
nmap-7.80p0 scan ports and fingerprint stack of network hosts
opus-tools-0.2  encode, inspect, and decode Opus files
p11-kit-0.23.22p0   library for loading and enumerating PKCS#11 modules
p7zip-16.02p6   file archiver with high compression ratio
pango-1.48.4    

Re: ssh authlog: Failed none for invalid user

[Jordan Geoghegan]

On 8/10/21 1:30 AM, Darren Tucker wrote:
> On Tue, 10 Aug 2021 at 09:06, Jordan Geoghegan  <mailto:jor...@geoghegan.ca>> wrote:
>
> Hello,
>
> I was hoping somebody could set me straight here. On one of my machines I 
> have a number of entries in my /var/log/authlog file that look like this:
>
>     Failed none for invalid user admin from 14.239.50.255 port 51796
>
> The machine has been being hammered with SSH bruteforce attempts and I 
> noticed that "Failed none" entry popping up frequently.
>
> What exactly does "Failed none" mean here in this in this context?
>
>
> It's the attempted authentication method, and it's normal behaviour.
>
> The SSH protocol has a number of authentication methods, for example 
> "password" and "publickey".    The client sends a message that says "I'd like 
> to authenticate via password using the password 'hunter2'" and the server 
> replies with either "yes that worked", or "nope" and a list of authentication 
> methods that it might accept.  Publickey authentication has a couple of extra 
> steps but works in a similar way.
>
> The protocol also specifies a "none" [0] authentication method, which will 
> succeed if the server requires no further authentication (eg in OpenSSH, if 
> PermitEmptyPassword is set and the account does not have a password).  Many 
> SSH clients including OpenSSH's start by asking for "none" authentication 
> then, if that doesn't work, use the list of possible authentication methods 
> to decide what to do next.  This is what you're seeing.
>
> When I last looked, the bulk of the password guessing bots just sent a single 
> "password" auth method and if it doesn't work, disconnect.  Apparently the 
> bots you're seeing behave a bit more like other clients.
>
> [0] https://datatracker.ietf.org/doc/html/rfc4252#section-5.2 
> <https://datatracker.ietf.org/doc/html/rfc4252#section-5.2>
>
> -- 
> Darren Tucker (dtucker at dtucker.net <http://dtucker.net>)
> GPG key 11EAA6FA / A86E 3E07 5B19 5880 E860  37F4 9357 ECEF 11EA A6FA (new)
>     Good judgement comes with experience. Unfortunately, the experience
> usually comes from bad judgement.

Hi Darren,

Thank you for that excellent, detailed answer - much appreciated!

Regards,

Jordan

ssh authlog: Failed none for invalid user

[Jordan Geoghegan]

Hello,

I was hoping somebody could set me straight here. On one of my machines I have 
a number of entries in my /var/log/authlog file that look like this:

    Failed none for invalid user admin from 14.239.50.255 port 51796

The machine has been being hammered with SSH bruteforce attempts and I noticed 
that "Failed none" entry popping up frequently.

What exactly does "Failed none" mean here in this in this context?

Any insight would be greatly appreciated as my Google-fu has failed me in my 
search for an answer.

Regards,

Jordan

Re: Another potential awk or xargs bug?

[Jordan Geoghegan]

On 4/15/21 7:49 AM, Otto Moerbeek wrote:
> On Thu, Apr 15, 2021 at 04:29:17PM +0200, Christian Weisgerber wrote:
>
>> Jordan Geoghegan:
>>
>>> --- /tmp/bad.txt  Wed Apr 14 21:06:51 2021
>>> +++ /tmp/good.txt  Wed Apr 14 21:06:41 2021
>> I'll note that no characters have been lost between the two files.
>> Only the order is different.
>>
>>> The only thing that changed between these runs was me using either xargs -P 
>>> 1 or -P 2.
>> What do you expect?  You run two processes in parallel that write
>> to the same file.  Obviously their output will be interspersed in
>> unpredictable order.
>>
>> You seem to imagine that awk's output is line-buffered.  But when
>> it writes to a pipe or file, its output is block-buffered.  This
>> is default stdio behavior.  Output is written in block-size increments
>> (16 kB in practice) without regard to lines.  So, yes, you can end
>> up with a fragment from a line written by process #1, followed by
>> lines from process #2, followed by the remainder of the line from
>> #1, etc.
>>
>> -- 
>> Christian "naddy" Weisgerber  na...@mips.inka.de
>>
> Right, a fflush() call after the printf makes the issue go away, but
> only since awk is being nice and issues a single write call for that
> single printf. Since awk afaik does not give such a guarantee, it is
> better to have each parallel invocation write to a separate file and
> then cat them together after all the awk runs are done.
>
>   -Otto

Hello Christian and Otto,

Thank you for setting me straight. The block vs line buffering issue should 
have been obvious to me. What got me confused was that this solution worked 
well, for a long time - until it didn't. One would assume that it would 
consistently mangle output...

While fflush does seem to fix the issue, I wanted to explore your suggestion 
Otto of writing to a temporary file from within awk.

Is something like the following a sane approach to safely generating temporary 
files from within awk?:

BEGIN{ cmd = "mktemp -q /tmp/workdir/tmp.XXX" ; if( ( cmd | getline result 
) > 0 ) TMPFILE = result ; else exit 1 }

Unless I'm missing something obvious, It seems there is no way to capture both 
the stdout and return code of an external command from within awk. My 
workaround solution to error check the call to mktemp here is to abort if 
mktemp returns no data. Is this sane?

Regards,

Jordan

Another potential awk or xargs bug?

[Jordan Geoghegan]

Hello,

I've found some very interesting behaviour when subjecting various awk 
implementations to some very specific circumstances.

I'm basically looking for a sanity check here to confirm if I'm just wildly 
flailing, or if I am indeed onto something here.

Here's my situation:

When parsing some RIR data in parallel using awk with xargs, I seem to have 
found a way to reliable lose and/or mangle output with parallel xargs. My 
google-fu seems to be failing me. I understand that xargs does not buffer 
output and that lines may arrive out of order, but in this case I am reliably 
and reproducibly losing data and receiving mangled output. But wait, it gets 
stranger.

I don't want to lose you guys here with a long winded explanation, so I'm going 
to show you a diff that shows reproducibly mangled output when using xargs in 
parallel mode:

--- /tmp/bad.txt  Wed Apr 14 21:06:51 2021
+++ /tmp/good.txt  Wed Apr 14 21:06:41 2021
@@ -1,5 +1,3 @@
-267386
-A264890
 AS262399
 AS262400
 AS262401
@@ -1774,6 +1772,7 @@
 AS264887
 AS264888
 AS264889
+AS264890
 AS264891
 AS264892
 AS264893
@@ -3552,6 +3551,7 @@
 AS267383
 AS267384
 AS267385
+AS267386
 AS267387
 AS267388
 AS267389
@@ -4220,6 +4220,7 @@
 AS268318
 AS268319
 AS268320
+AS268320
 AS268321
 AS268321
 AS268323
@@ -7785,6 +7786,7 @@
 AS270633
 AS270633
 AS270634
+AS270634
 AS270635
 AS270635
 AS270636
@@ -10277,5 +10279,3 @@
 AS46210
 AS46280
 AS46280
-ASAS268320
-ASS270634

The only thing that changed between these runs was me using either xargs -P 1 
or -P 2.

To allow folks to follow along with me at home, I've included the two files 
(gzipped for politeness) I used to trigger this behaviour.

Once you've extracted the attached text files into your working directory, 
here's a snippet that should reproduce my issue:

$ printf 'BR\nCA\n' > cc.txt

$ find . -type f -name "[12].txt" -print0 | xargs -0 -n 1 -P 2 -- awk -F '|' 
'NR==FNR { A[$1]=1 ; next } $1 in A && $2 == "asn" { printf("AS%s\n", $3) }' 
cc.txt

What does this 1 liner do, well it's supposed to slurp the country codes 
specified in cc.txt into an array where we then check the first field of each 
row of the RIR data against. If the first field matches a country code in the 
array and the second field indicates that this row is an ASN record, then we 
print the 3rd field prepended with 'AS'. As you can see, if you grep the output 
of the above command for the string "ASAS", "ASS" or 'A2' you should see some 
mangled ASNs. If you change "-P 2" to "-P 1" this mangling will not occur.

Here's where things get very weird. While parsing this data (as part of a 
larger dataset comprising an aggregation of all the registrar delegation 
statistics) I've been using this snippet for a while to quickly fetch ASN 
records. It is not until I have BOTH the BR and CA country codes in the array 
that I can trigger this bug. I can have any number of country codes in the 
array, but if Brazil AND Canada happen to be specified in the array, then I get 
mangled output, but ONLY if executed with parallel xargs. This reproducibly 
happens when using awk, gawk or mawk. To further melt your brain, this 
behaviour has NOT been observed when using goawk, a POSIX compliant awk 
implementation written in go.

Just to prove my point, here's me testing the hash outputs between various awk 
implementations with my above 1 liner:

$ find . -type f -name "[12].txt" -print0 | xargs -0 -n 1 -P 2 -- awk -F '|' 
'NR==FNR { A[$1]=1 ; next } $1 in A && $2 == "asn" { printf("AS%s\n", $3) }' 
cc.txt | sort | md5
    2a20f44ce6a23d5c49b05b9f2689ef93

$ find . -type f -name "[12].txt" -print0 | xargs -0 -n 1 -P 1 -- awk -F '|' 
'NR==FNR { A[$1]=1 ; next } $1 in A && $2 == "asn" { printf("AS%s\n", $3) }' 
cc.txt | sort | md5
    9ab3dbfbff5746f059cdb35221ff73b1
---
$ find . -type f -name "[12].txt" -print0 | xargs -0 -n 1 -P 2 -- mawk -F '|' 
'NR==FNR { A[$1]=1 ; next } $1 in A && $2 == "asn" { printf("AS%s\n", $3) }' 
cc.txt | sort | md5
    2a20f44ce6a23d5c49b05b9f2689ef93

$ find . -type f -name "[12].txt" -print0 | xargs -0 -n 1 -P 1 -- mawk -F '|' 
'NR==FNR { A[$1]=1 ; next } $1 in A && $2 == "asn" { printf("AS%s\n", $3) }' 
cc.txt | sort | md5 >
    9ab3dbfbff5746f059cdb35221ff73b1
---
$ find . -type f -name "[12].txt" -print0 | xargs -0 -n 1 -P 2 -- 
~/go/bin/goawk -F '|' 'NR==FNR { A[$1]=1 ; next } $1 in A && $2 == "asn" { 
printf("AS%s\n", $3) }' cc.txt | sort | md>
    9ab3dbfbff5746f059cdb35221ff73b1

$ find . -type f -name "[12].txt" -print0 | xargs -0 -n 1 -P 1 -- 
~/go/bin/goawk -F '|' 'NR==FNR { A[$1]=1 ; next } $1 in A && $2 == "asn" { 
printf("AS%s\n", $3) }' cc.txt | sort | md
    9ab3dbfbff5746f059cdb35221ff73b1

I've racked my brain and the internet for hours, I've tested and toiled, and 
I'm left thoroughly perplexed. I now humbly ask the fine folks here in OpenBSD 
Land for guidance, insight or suggestions.

As always, is this a bug, or am I holding it wrong?

Regards,

Jordan



1.txt.gz
Description: 

Re: 6.9 Current amd64 xfce seems to freeze and not respond to mouse clicks or keystrokes

[Jordan Geoghegan]

On 4/10/21 2:48 PM, gwes wrote:
>
>
> On 4/10/21 5:22 PM, Tom Smyth wrote:
>> Hello,
>>
>> 1) issue does not occur with fvwm or with chrome running in fvwm
>>
>> so the issue seems to be confined to xfce, and I was running  just 1
>> xfce terminal session
>> 2) (so the issue is not related to chromium)
>>
>> Thanks
>>
>>
>> O
>> -- 
>> Kindest regards,
>> Tom Smyth.
>>
> Hi Tom,
> Some application that you can't see is grabbing focus and not letting go.
>
> On another OS using xfce (XUbuntu) the screensaver sometimes causes something
> extremely similar. Mouse cursor moves but nothing else responds.
> The workaround is to use control-alt-F1 to get a
> plain console and ps -ax | grep screen then doas kill .
>
> If it's not a screensaver it's almost always a second browser copy.
> I just find likely greedy candidates in the ps and kill until the problem 
> goes away.
>
> If the X server won't let you use control-alt-Fx to change screens you'll
> have to ssh in.
>
> Geoff Steckel
>

That xfce screen saver issue also seems to exists on OpenBSD 6.8, as I've 
triggered it a number of times on my desktop. I've found that even with the 
locked screen (mouse can move and nothing else as described) I can still  enter 
my password to unlock the screen saver, and then things usually work as 
expected -- if not, switching terminals with ctrl+alt+f1 and back with 
ctrl+alt+f5 seems to do the trick once the screen saver has been unlocked with 
my users password, with no need to kill the screen saver process.

Regards,

Jordan

Re: blacklistd analogue

[Jordan Geoghegan]

On 3/24/21 11:48 AM, Peter Nicolai Mathias Hansteen wrote:
>> 24. mar. 2021 kl. 19:33 skrev jeanpierre 
>> :
>>
>> Does there exist an OpenBSD analogue for FreeBSD's blacklistd daemon?
>>
>> For the sake of completeness: blacklistd is a daemon that, using pf
>> anchors, blocks connections from abusive hosts to parctiular services
>> (e.g. sshd) until they start behaving themselves again.
>>
>> I find it very useful for timming down log files.
> Not in the base system but you might want to take a peek at pf-badhosts 
> (described among other places in this OpenBSD Journal article 
> https://undeadly.org/cgi?action=article;sid=20210119113425 
> ) which should be 
> fairly easy to adapt to using more or other sources such as the bsdly.net 
>  feed maintained mainly by kind robots under supervision 
> by yours truly (see 
> https://bsdly.blogspot.com/2018/08/badness-enumerated-by-robots.html 
>  and 
> links therein and in the proximity)
>
> Cheers,
> Peter
>
> —
> Peter N. M. Hansteen, member of the first RFC 1149 implementation team
> http://bsdly.blogspot.com/ http://www.bsdly.net/ http://www.nuug.no/
> "Remember to set the evil bit on all malicious network traffic"
> delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.
>
>

I use Peter's "bruteforcers" list on my personal pf-badhost installs. 
pf-badhost will happily parse and ingest the IPv4 and IPv6 data in Peter's 
blocklists.

Just use the '-l' option to include an additional URL. Something like this 
should work to include his lists:

    $ pf-badhost -O openbsd -l 'https://www.bsdly.net/~peter/bruteforcers.txt'

I intend to include this among the default lists in the next release -- the 
only reason it wasn't included in v0.5 was because I discovered it too late.

Happy bot blocking!

Regards,

Jordan

Re: Attempting to use Brother DCP-L2510D printer on BSD

[Jordan Geoghegan]

On 2/24/21 9:43 AM, Sylvain S wrote:
> Hello,
>
> I see that, if I understand correctly, you have been
> able to make this printer work on ArchLinux from the
> officially provided RPM driver package.
> https://aur.archlinux.org/packages/brother-dcp-l2510d/
>
> May I ask, what kind of porting work did that require ?
> I have acquired this model from Brother and have not been
> successful at making it work/print yet.
> For reference, here is the thread on the French community forums
> https://forum.openbsd.fr.eu.org/showthread.php?tid=2846
>
> I also note that the printer is absent from the foomatic
> database maintained by Fedora. This seems strange as a
> driver package is available for their distribution format,
> but perhaps I am missing something here ?
> https://src.fedoraproject.org/repo/extras/foomatic-db/
>
> Finally, I notice from the documentation (to the mailing list's attention) :
>
> The Foomatic framework supports direct printing which does not rely on CUPS 
> (nor
> any spooler).
>
> Complete documentation can be found at:
>     
> https://wiki.linuxfoundation.org/openprinting/database/nospoolerdocumentation
>
> I would gladly do that, but am still clueless as of now
>
> Thanks to anyone who would help
>

It feels like we repeat this thread every couple months...

If your printer supports postscript (as yours and most other Brother brand 
laser printers seem to) you can use something like this in your /etc/printcap 
file to print postscript data directly to the printer:
...
lp|remote line printer:\
    
:lp=:rm=172.20.200.5:rp=lp:sd=/var/spool/output/lpd:lf=/var/log/lpd-errs:
...
Then check out stuff like the 'pdftops' utility etc for converting documents to 
postscript. With a bit of fiddling, LibreOffice can be configured to print raw 
postscript to the device as well. No drivers or other garbage needed. Everybody 
always wants to connect to their printer over USB, just don't -- its annoying. 
If you get your postscript printer setup on the network like this, then you can 
move forward into broad sunlit uplands.

Printing truly can be as easy as '$ lpr mydoc.ps'

Regards,

Jordan

Re: relayd + pfsync

[Jordan Geoghegan]

On 2/1/21 8:20 PM, Kapetanakis Giannis wrote:
> On 02/02/2021 05:18, Jordan Geoghegan wrote:
>> Hello,
>>
>> I had a question about using relayd with pfsync.
>>
>> I have a small gateway/load-balancer set up with relayd, carp and pfsync 
>> plus BGPd for IP failover, and everything is working great. I was pleasantly 
>> surprised at how easy it was to get pfsync tunnelled over wireguard. Things 
>> failover perfectly, and I'm happy as a clam.
>>
>> I however do have a question about some pfsync/relayd details that I'm not 
>> fully clear on:
>>
>> With all the plumbing being done with relayd and all the associated 
>> TCP/TLS/HTTP(s) checks it's doing, it ends up setting up and tearing down a 
>> decent number of connections on a recurring basis. I know in PF you can use 
>> the "no-sync" keyword to prevent states created by certain rules from being 
>> synced across the wire, but I haven't found a way to do this with 
>> rules/states generated by relayd.
>>
>> It's probably largely irrelevant in the grand scheme of things, but I found 
>> it slightly irritating having hundreds or thousands of state table entries 
>> experiencing constant churn while being synced over the wire. Having the 
>> noise from the relayd connectivity checks syncing back and forth makes using 
>> tcpdump on a pfsync interface much less convenient. All these state table 
>> entries will never be used should the machine fail-over, as all the 
>> connectivity checks are initiated from the local IP address, rather that the 
>> CARP address.
>>
>> So I guess what I'm trying to ask is: Is there a way to have relayd not sync 
>> it's TCP/TLS/etc connectivity checks via pfsync?
>>
>> I was hoping to get a sanity check here so I can confirm weather or not I'm 
>> totally off base here.
>>
>> I currently have "keep state (no-sync)" peppered throughout my config for 
>> rules I want excluded from pfsync, as the pf config is quite simple. Maybe 
>> I'm missing something obvious, but is there a "sync" option? ie the ability 
>> to manually specify exactly which rules/states you want synced?
>>
>> Would some sort of rule like "pass out on $int_if proto tcp to any user 
>> _relayd keep state (no-sync)" do what I want, or would that also catch the 
>> traffic I'm trying to load balance as well?
>>
>> Any insight or advice would be much appreciated.
>>
>> P.S  Sorry for the wall of text
>>
>> Regards,
>>
>> Jordan
>
> Hi,
>
> As you said, you can use the no-sync.
>
> Relayd checks don't create any pf rules. Only the listen creates rules
> pfctl -sr -a'relayd/ldap'
>
> pass in quick on rdomain 0 inet proto tcp from any to x.x.x.x port = 636 
> flags S/SA keep state (tcp.established 4200) tag RELAYD_ldap rdr-to  
> port 1636 least-states sticky-address
>
> local checks from LB to hosts can have the no-sync.
> I have these in my config
>
> # checks from LB
> pass out quick on $ldap_if proto tcp from ($ldap_if) to ($ldap_if:network) 
> port {1389, 1636} keep state (no-sync)
>
> Maybe you have another rule (out on $ldap_if) before, that allows the traffic?
>
> I handle incoming traffic (to LB) with pftag on relayd.conf and I 
> specifically allow them on out direction.
>
> # client rules
> pass out quick on $ldap_if tagged RELAYD_ldap keep state (tcp.established 
> 4200)
>
> G
>
>

Hello,

Thanks for the sanity check, I've got everything working as expected now!

I need to learn to step away from the keyboard when I'm getting tired and 
frustrated. The answer seems obvious in retrospect, but at least its figured 
out now.

Regards,

Jordan

relayd + pfsync

[Jordan Geoghegan]

Hello,

I had a question about using relayd with pfsync.

I have a small gateway/load-balancer set up with relayd, carp and pfsync plus 
BGPd for IP failover, and everything is working great. I was pleasantly 
surprised at how easy it was to get pfsync tunnelled over wireguard. Things 
failover perfectly, and I'm happy as a clam.

I however do have a question about some pfsync/relayd details that I'm not 
fully clear on:

With all the plumbing being done with relayd and all the associated 
TCP/TLS/HTTP(s) checks it's doing, it ends up setting up and tearing down a 
decent number of connections on a recurring basis. I know in PF you can use the 
"no-sync" keyword to prevent states created by certain rules from being synced 
across the wire, but I haven't found a way to do this with rules/states 
generated by relayd.

It's probably largely irrelevant in the grand scheme of things, but I found it 
slightly irritating having hundreds or thousands of state table entries 
experiencing constant churn while being synced over the wire. Having the noise 
from the relayd connectivity checks syncing back and forth makes using tcpdump 
on a pfsync interface much less convenient. All these state table entries will 
never be used should the machine fail-over, as all the connectivity checks are 
initiated from the local IP address, rather that the CARP address.

So I guess what I'm trying to ask is: Is there a way to have relayd not sync 
it's TCP/TLS/etc connectivity checks via pfsync?

I was hoping to get a sanity check here so I can confirm weather or not I'm 
totally off base here.

I currently have "keep state (no-sync)" peppered throughout my config for rules 
I want excluded from pfsync, as the pf config is quite simple. Maybe I'm 
missing something obvious, but is there a "sync" option? ie the ability to 
manually specify exactly which rules/states you want synced?

Would some sort of rule like "pass out on $int_if proto tcp to any user _relayd 
keep state (no-sync)" do what I want, or would that also catch the traffic I'm 
trying to load balance as well?

Any insight or advice would be much appreciated.

P.S  Sorry for the wall of text

Regards,

Jordan

Re: File this bug, or not?

[Jordan Geoghegan]

On 1/18/21 2:47 PM, Eric Zylstra wrote:
> Misc,
>
> I’ve set up a 6 drive RAID-5.  Just for the experience of degrading and 
> rebuilding the RAID, I popped a drive out.  Within a few seconds the machine 
> kerneled and dropped into ddb.  Is there any chance this would be expected 
> considering the machine’s SATA is not hot-swappable?
>
> I’m looking into setting up a serial connection so I can capture the debut 
> output (I already have photos of the traces for all 8 CPU, but would like to 
> give serial output instead).  I would not file a report if this behavior 
> falls into “not great, but expected”.
>
> Thanks,
>
> EZ
>

Just thought I'd chip in here too FWIW:

I've never successfully hot swapped a drive with OpenBSD before. I have 
hardware that does it fine on Linux, but fails on OpenBSD. I haven't caused the 
kernel to panic when pulling a drive, but the OS fails to detect any newly 
attached SATA or SAS drives. It's certainly caused some frustration when trying 
to rebuild a RAID array on a production machine. Maybe I just have wonky 
hardware, but I've tried this on a number of releases, on several different 
pieces of hardware, on several different arches. I have no solution to offer, 
just thought I'd share my experience with hot swapping drives on OpenBSD.

Regards,

Jordan

pf-badhost and unbound-adblock v0.5 released

[Jordan Geoghegan]

Hey folks,

Since I've been getting a lot of emails about this, I just thought I'd drop a 
line here and let you know that I've released pf-badhost and unbound-adblock 
version 0.5.

You can see the release pages here:

pf-badhost is a fast, in-kernel, bi-directional network filtering utility 
powered by the PF firewall.
https://geoghegan.ca/pfbadhost.html

unbound-adblock is a fast, flexible, easy to use DNS firewall utility.
https://geoghegan.ca/unbound-adblock.html

This will be the last time I advertise my wares on here as I have established a 
notification service -- just send an email to "annou...@geoghegan.ca" to 
subscribe.

One new feature that I would like to highlight is that unbound-adblock now 
supports florian@'s excellent unwind(8) resolver as an optional backend. I've 
been running this on my laptop for about a month now and it's working quite 
nicely.

Thank you to everyone in the the OpenBSD community who have on many occasions, 
kindly assisted with development in one way or the other -- I am truly grateful.

Regards,

Jordan Geoghegan

Re: cmp(1) '-s' flag ignoring byte offset argument?

[Jordan Geoghegan]

On 1/9/21 1:59 AM, Otto Moerbeek wrote:
> On Sat, Jan 09, 2021 at 12:05:31AM -0800, William Ahern wrote:
>
>> On Fri, Jan 08, 2021 at 07:09:01PM -0800, Jordan Geoghegan wrote:
>>> Hey folks,
>>>
>>> I've noticed some surprising behaviour from cmp(1) when using the '-s'
>>> flag.
>>>
>>> It appears that cmp -s is ignoring the byte offset arguments I'm giving
>>> it.
>> 
>>> Not sure what to make of this, I noticed this same behaviour on
>>> DragonflyBSD and FreeBSD, so maybe I'm just missing something obvious.
>>> This certainly caused some frustration before I figured out what was going
>>> on.
>> The bug seems to be in the short-circuit optimization for regular files[1]:
>>
>> void
>>   c_regular(int fd1, char *file1, off_t skip1, off_t len1,
>>   int fd2, char *file2, off_t skip2, off_t len2)
>>   {
>>  u_char ch, *p1, *p2;
>>  off_t byte, length, line;
>>  int dfound;
>>   
>>  if (sflag && len1 != len2)
>>  exit(1);
>>   
>>  if (skip1 > len1)
>>  eofmsg(file1);
>>  len1 -= skip1;
>>  if (skip2 > len2)
>>  eofmsg(file2);
>>  len2 -= skip2;
>>
>> The short-circuit should probably be moved below the subsequent chunk of
>> code (i.e. below `len2 -= skip2`). The eofmsg function already obeys sflag,
>> so it'll be quiet.[2] Doing this works for me. See patch at end of message.
>>
>> Interestingly, DragonflyBSD and FreeBSD already do it this way[3][4], yet I
>> can confirm FreeBSD still has the problem. (DragonflyBSD has nearly
>> identical code.) But that implementation duplicates the short-circuit, along
>> with the bug of not accounting for skip1 and skip2, in cmp.c as part of
>> implementing the -z flag[5]:
>>
>>  if (special)
>>  c_special(fd1, file1, skip1, fd2, file2, skip2);
>>  else {
>>  if (zflag && sb1.st_size != sb2.st_size) {
>>  if (!sflag)
>>  (void) printf("%s %s differ: size\n",
>>  file1, file2);
>>  exit(DIFF_EXIT);
>>  }
>>  c_regular(fd1, file1, skip1, sb1.st_size,
>>  fd2, file2, skip2, sb2.st_size);
>>  }
>>  exit(0);
>>
>> It appears that the June 20, 2000 fix to the short-circuit in regular.c
>> wasn't recognized during the July 14, 2000 -z feature addition.[6][7]
>>
>> [1] https://cvsweb.openbsd.org/src/usr.bin/cmp/regular.c?rev=1.12
>> [2] https://cvsweb.openbsd.org/src/usr.bin/cmp/misc.c?rev=1.7
>> [3] 
>> https://gitweb.dragonflybsd.org/dragonfly.git/blob/4d4f84f:/usr.bin/cmp/regular.c
>> [4] 
>> https://svnweb.freebsd.org/base/head/usr.bin/cmp/regular.c?revision=344551
>> [5] 
>> https://svnweb.freebsd.org/base/head/usr.bin/cmp/cmp.c?revision=344551=markup#l193
>> [6] 
>> https://svnweb.freebsd.org/base/head/usr.bin/cmp/regular.c?revision=61883=markup
>> [7] 
>> https://svnweb.freebsd.org/base/head/usr.bin/cmp/cmp.c?view=markup=63157
>>
>> --- regular.c6 Feb 2015 23:21:59 -   1.12
>> +++ regular.c9 Jan 2021 07:51:13 -
>> @@ -51,15 +51,15 @@ c_regular(int fd1, char *file1, off_t sk
>>  off_t byte, length, line;
>>  int dfound;
>>  
>> -if (sflag && len1 != len2)
>> -exit(1);
>> -
>>  if (skip1 > len1)
>>  eofmsg(file1);
>>  len1 -= skip1;
>>  if (skip2 > len2)
>>  eofmsg(file2);
>>  len2 -= skip2;
>> +
>> +if (sflag && len1 != len2)
>> +exit(1);
>>  
>>  length = MINIMUM(len1, len2);
>>  if (length > SIZE_MAX) {
>>
> I came to the same diff independently. In the meantime it has been committed.
>
>   -Otto
>

Hi Otto and William,

Thanks for confirming that this is a bug, and also for the fixes!

Regards,

Jordan

cmp(1) '-s' flag ignoring byte offset argument?

[Jordan Geoghegan]

Hey folks,

I've noticed some surprising behaviour from cmp(1) when using the '-s' flag.

It appears that cmp -s is ignoring the byte offset arguments I'm giving it.

I don't want to waste time babbling, so here's an example snippet to show what 
I'm talking about:

#!/bin/sh

echo 'my line' > /tmp/1.txt
echo 'my other line' >> /tmp/1.txt
echo 'same same' >> /tmp/1.txt

echo 'my differnt line' > /tmp/2.txt
echo 'my other different line' >> /tmp/2.txt
echo 'same same' >> /tmp/2.txt

# Determine byte offsets (we only want to compare lines >= 3)
offset1="$(head -2 /tmp/1.txt | wc -c)"
offset2="$(head -2 /tmp/2.txt | wc -c)"

# Compare files and show exit code
cmp /tmp/1.txt /tmp/2.txt "$offset1" "$offset2"
printf '\nReturn code = %s\n' "$?"

cmp -s /tmp/1.txt /tmp/2.txt "$offset1" "$offset2"
printf '\nReturn code with "-s" = %s\n' "$?"

As you can see, 'cmp -s' returns an exit code of '1', unlike cmp without the 
'-s' which returns '0'.

Not sure what to make of this, I noticed this same behaviour on DragonflyBSD 
and FreeBSD, so maybe I'm just missing something obvious. This certainly caused 
some frustration before I figured out what was going on.

Regards,

Jordan

An ode to OpenBSD

[Jordan Geoghegan]

This isn't actually an ode, as I can't rhyme things good, but I just 
wanted to thank the OpenBSD folks for their fantastic engineering.


I had a major power outage and network disruption at my hosting provider 
yesterday, and all but one of my relayd load balancers was knocked out, 
as well as much of my httpd cluster.


Even while Zabbix was blowing up my phone with alerts, thanks to the 
magic of CARP+pfsync+relayd, all my services remained available, and a 
disaster was largely turned into a non-event.


Thank you for creating the greatest software project of all time.

Regards,

Jordan

Re: OpenBSD Monitor Sleep No Response

[Jordan Geoghegan]

On 12/20/20 6:26 PM, ben wrote:

Hello, misc;

I've been having an issue with my OpenBSD install, specifically when the system
turns off the monitor after a period of no use. After the monitor goes blank I
can't use the start using the machine and must restart, that is after keypress
and mouse movement the system does not show anything on the monitor. I suspect
something is wrong due to hardware. I've checked the logs, nothing seems to be
off. I've turned off apmd as to not interfer with power management and still no
response after the monitor goes to sleep.

Here's a list of the hardware:

  - AMD Ryzen 5 3400G Processor
  - Asus Prime B550M-A/CSM Motherboard
  - Radeon RX 580 POLARIS10 GPU

Has anyone else experience any issues with like this? Is there still no support
for polaris GPUs? Thank you in advance.


Ben Raskin



I've had this issue as well with a Ryzen machine with an R9290x graphics 
card (ie not supported by AMDGPU). My monitor is using DisplayPort but 
the same behaviour is present when using HDMI monitors as well.


I've found the only way I can get output to the screen again is by 
switching consoles with CTRL+ALT+F1 and then switching back to the xorg 
console with CTRL+ALT+F5.


Not sure what the cause is, but I've always assumed it was an issue the 
the radeon driver.


Regards,

Jordan

Re: Potential dig bug?

[Jordan Geoghegan]

On 12/16/20 11:19 PM, Otto Moerbeek wrote:

On Wed, Dec 16, 2020 at 02:37:19PM -0800, Jordan Geoghegan wrote:


Hi folks,

I've found some surprising behaviour in the 'dig' utility. I've noticed that
dig doesn't seem to support link local IPv6 addresses. I've got unbound
listening on a link local IPv6 address on my router and all queries seem to
be working. I'm advertising this DNS info with rad, and I confirmed with
tcpdump that my devices such as iPhones, Macs, Windows, Linux desktops etc
are all properly querying my unbound server over IPv6.

dhclient doesn't seem to allow you to specify an IPv6 address in it's
'supersede'  options, so I manually edited my OpenBSD desktops resolv.conf
to specify the IPv6 unbound server first. Again, I confirmed with tcpdump
that my desktop was properly querying the unbound server over IPv6 (ie
Firefox, ping, ssh etc all resolved domains using this server).

I used 'dig' to make a query, and I noticed it was ignoring my link local
IPv6 nameserver in my resolv.conf. I'll save you guys the long form Ted talk
here and just make my point:

$ cat resolv.conf
    nameserver fe80::f29f:c2ff:fe17:b8b2%em0
    nameserver 2606:4700:4700::
    lookup file bind
    family inet6 inet4

$ dig google.ca
    [snip]
    ;; Query time: 12 msec
    ;; SERVER: 2606:4700:4700::#53(2606:4700:4700::)
    [snip]

There's a bit of a delay as it waits for a time out, and then it falls back
to the cloudflare IPv6 server.

I tried specifying the server with '@' as well as specifying source
IP/interface with '-I' to no avail. It seems dig really doesn't like the
'fe80::%em0' notation, as  '@' and '-I' worked fine when used without a
link-local address.

Is this a bug or a feature? Am I just doing something stupid? Any insight
would be appreciated.

I think it is a bug and I can reproduce. Will invesigate deeper later.

-Otto



Hi Otto,

Thanks for looking into this! I took Bodie's advice and tested nslookup 
and host, and they both seem to have the same behaviour as dig.


Regards,

Jordan

Re: Potential dig bug?

[Jordan Geoghegan]

On 12/16/20 2:37 PM, Jordan Geoghegan wrote:

Hi folks,

I've found some surprising behaviour in the 'dig' utility. I've 
noticed that dig doesn't seem to support link local IPv6 addresses. 
I've got unbound listening on a link local IPv6 address on my router 
and all queries seem to be working. I'm advertising this DNS info with 
rad, and I confirmed with tcpdump that my devices such as iPhones, 
Macs, Windows, Linux desktops etc are all properly querying my unbound 
server over IPv6.


dhclient doesn't seem to allow you to specify an IPv6 address in it's 
'supersede'  options, so I manually edited my OpenBSD desktops 
resolv.conf to specify the IPv6 unbound server first. Again, I 
confirmed with tcpdump that my desktop was properly querying the 
unbound server over IPv6 (ie Firefox, ping, ssh etc all resolved 
domains using this server).


I used 'dig' to make a query, and I noticed it was ignoring my link 
local IPv6 nameserver in my resolv.conf. I'll save you guys the long 
form Ted talk here and just make my point:


$ cat resolv.conf
   nameserver fe80::f29f:c2ff:fe17:b8b2%em0
   nameserver 2606:4700:4700::
   lookup file bind
   family inet6 inet4

$ dig google.ca
   [snip]
   ;; Query time: 12 msec
   ;; SERVER: 2606:4700:4700::#53(2606:4700:4700::)
   [snip]

There's a bit of a delay as it waits for a time out, and then it falls 
back to the cloudflare IPv6 server.


I tried specifying the server with '@' as well as specifying source 
IP/interface with '-I' to no avail. It seems dig really doesn't like 
the 'fe80::%em0' notation, as  '@' and '-I' worked fine when used 
without a link-local address.


Is this a bug or a feature? Am I just doing something stupid? Any 
insight would be appreciated.


Regards,

Jordan


Sorry for the double mail, I hit send too early...

Woops, I failed to make the key point here:

I checked with tcpdump and confirmed that dig does not even attempt to 
query the IPv6 link local DNS server, even though it reports a timeout - 
ie dig sends no traffic over the wire destined to that address:


; <<>> dig 9.10.8-P1 <<>> @fe80::f29f:c2ff:fe17:b8b2%em0 google.ca
; (1 server found)
;; global options: +cmd
;; connection timed out; no servers could be reached

Regards,
Jordan

Potential dig bug?

[Jordan Geoghegan]

Hi folks,

I've found some surprising behaviour in the 'dig' utility. I've noticed 
that dig doesn't seem to support link local IPv6 addresses. I've got 
unbound listening on a link local IPv6 address on my router and all 
queries seem to be working. I'm advertising this DNS info with rad, and 
I confirmed with tcpdump that my devices such as iPhones, Macs, Windows, 
Linux desktops etc are all properly querying my unbound server over IPv6.


dhclient doesn't seem to allow you to specify an IPv6 address in it's 
'supersede'  options, so I manually edited my OpenBSD desktops 
resolv.conf to specify the IPv6 unbound server first. Again, I confirmed 
with tcpdump that my desktop was properly querying the unbound server 
over IPv6 (ie Firefox, ping, ssh etc all resolved domains using this 
server).


I used 'dig' to make a query, and I noticed it was ignoring my link 
local IPv6 nameserver in my resolv.conf. I'll save you guys the long 
form Ted talk here and just make my point:


$ cat resolv.conf
   nameserver fe80::f29f:c2ff:fe17:b8b2%em0
   nameserver 2606:4700:4700::
   lookup file bind
   family inet6 inet4

$ dig google.ca
   [snip]
   ;; Query time: 12 msec
   ;; SERVER: 2606:4700:4700::#53(2606:4700:4700::)
   [snip]

There's a bit of a delay as it waits for a time out, and then it falls 
back to the cloudflare IPv6 server.


I tried specifying the server with '@' as well as specifying source 
IP/interface with '-I' to no avail. It seems dig really doesn't like the 
'fe80::%em0' notation, as  '@' and '-I' worked fine when used without a 
link-local address.


Is this a bug or a feature? Am I just doing something stupid? Any 
insight would be appreciated.


Regards,

Jordan

Re: Another potential ksh bug?

[Jordan Geoghegan]

Hi Vadim,

Thanks for looking into this, all I ever wanted was an ancient pdksh bug 
fix for Christmas.


Regards,

Jordan



On 12/7/20 10:47 AM, Vadim Zhukov wrote:

For me, this is a definite bug. I've opted my students to fix this
bug, so unless there's a hurry, there must be a fix till the end of
December. :)

пн, 7 дек. 2020 г. в 07:43, Jordan Geoghegan :

Hello again,

I was playing around with ksh array syntax and its behaviour when set as
read-only. In my testing I noticed that ksh will allow you to overwrite
the first element of a read-only array. Example snippet:

#!/bin/ksh
arr[0]=val1
arr[1]=val2
readonly arr
echo "${arr[@]}"
arr=yikes
echo "${arr[@]}"

I tested a few other shells, and this bug does exists in the original
pdksh and is also present in zsh. This bug is not present in ksh93, mksh
or bash, where they abort when trying to modify the read-only array.

I don't have access to a proper ksh88 shell, but it would be nice if
someone could confirm its behaviour.

I was just hoping someone could confirm if this is intended behaviour,
or if it's a bug.

Regards,

Jordan

Re: Another potential ksh bug?

[Jordan Geoghegan]

On 12/7/20 10:14 PM, Noth wrote:


On 07/12/2020 05:41, Jordan Geoghegan wrote:

Hello again,

I was playing around with ksh array syntax and its behaviour when set 
as read-only. In my testing I noticed that ksh will allow you to 
overwrite the first element of a read-only array. Example snippet:


#!/bin/ksh
arr[0]=val1
arr[1]=val2
readonly arr
echo "${arr[@]}"
arr=yikes
echo "${arr[@]}"

I tested a few other shells, and this bug does exists in the original 
pdksh and is also present in zsh. This bug is not present in ksh93, 
mksh or bash, where they abort when trying to modify the read-only 
array.


I don't have access to a proper ksh88 shell, but it would be nice if 
someone could confirm its behaviour.


I was just hoping someone could confirm if this is intended 
behaviour, or if it's a bug.


Regards,

Jordan


Hi,

  I tries this with AMIX (UNIX SVR4 for Commodore Amiga) v2.1 which 
presumably is using ksh88 as it hasn't been updated since circa 1992 
and it does abort at the same place. Screenshot here: 
http://casper.nineinchnetworks.ch/images/kshtest.png .


Cheers,

Noth


Hi Noth,

Sorry for the late reply, I've been AFK for several days.

Thanks for testing that out and confirming ksh88's behaviour!

Regards,

Jordan

Another potential ksh bug?

[Jordan Geoghegan]

Hello again,

I was playing around with ksh array syntax and its behaviour when set as 
read-only. In my testing I noticed that ksh will allow you to overwrite 
the first element of a read-only array. Example snippet:


#!/bin/ksh
arr[0]=val1
arr[1]=val2
readonly arr
echo "${arr[@]}"
arr=yikes
echo "${arr[@]}"

I tested a few other shells, and this bug does exists in the original 
pdksh and is also present in zsh. This bug is not present in ksh93, mksh 
or bash, where they abort when trying to modify the read-only array.


I don't have access to a proper ksh88 shell, but it would be nice if 
someone could confirm its behaviour.


I was just hoping someone could confirm if this is intended behaviour, 
or if it's a bug.


Regards,

Jordan

Re: Potential ksh bug?

[Jordan Geoghegan]

On 11/16/20 10:36 PM, Bodie wrote:



On 17.11.2020 05:04, Jordan Geoghegan wrote:

Hello,

I'm not sure if this is a bug, or if it's just a pdksh thing, but I
stumbled upon some interesting behaviour when I was tinkering around
with quoting and using a poor mans array:

test=$(cat <<'__EOT'
# I'll choose not to close this quote
other_stuff
__EOT
)

echo "$test"


When I run this command on ash, dash, yash, bash, zsh or ksh93 I get
the following output:

# I'll choose not to close this quote
other_stuff

But when I run it on ksh from base or any pdksh derivative it throws
an error about an unclosed quote:

test.sh[8]: no closing quote

This snippet works on every POSIX-y shell in the ports tree, and fails
on every pdksh variant I tried, including on NetBSD and DragonflyBSD
as well.  I don't have the requisite esoteric knowledge regarding
pdksh's internal quoting logic, so I'm hoping one of the gurus here
can determine whether this is a bug or if I'm just doing something
annoying.

Any insight that can be provided would be much appreciated.



What exactly are you trying to achieve?

If you will look in sh(1) for 'Command expansion' then there are defined
rules and your form is not between them.

So error message about missing closing quote is actually proper behavior.

Plus I will not be so sure that bash is behaving properly:

$ bash -x test.sh
++ cat
+ test='
# I'\''ll choose not to close this quote
other_stuff'
+ echo '#' 'I'\''ll' choose not to close this quote other_stuff
# I'll choose not to close this quote other_stuff
$

Notice how one single quote is here "test='" abd second one is in
"other_stuff'"

You can compare results between various shells with ktrace/ktruss too

$ bash --version | head -1
GNU bash, version 5.0.18(1)-release (x86_64-unknown-openbsd6.8)
$

If you want to read the file then you do not need interaction and if
you want to get input from user then use 'read'

As well it is good idea to avoid reserved words as a names for 
variables ;-)

(test)


Regards,

Jordan




I'm not trying to achieve anything in particular, I was just testing out 
differences in behaviour between different shells. I just noticed that 
pdksh variants handled this differently than any other shell I tested in 
the ports tree.


Regards,

Jordan

Potential ksh bug?

[Jordan Geoghegan]

Hello,

I'm not sure if this is a bug, or if it's just a pdksh thing, but I 
stumbled upon some interesting behaviour when I was tinkering around 
with quoting and using a poor mans array:


test=$(cat <<'__EOT'
# I'll choose not to close this quote
other_stuff
__EOT
)

echo "$test"


When I run this command on ash, dash, yash, bash, zsh or ksh93 I get the 
following output:


# I'll choose not to close this quote
other_stuff

But when I run it on ksh from base or any pdksh derivative it throws an 
error about an unclosed quote:


test.sh[8]: no closing quote

This snippet works on every POSIX-y shell in the ports tree, and fails 
on every pdksh variant I tried, including on NetBSD and DragonflyBSD as 
well.  I don't have the requisite esoteric knowledge regarding pdksh's 
internal quoting logic, so I'm hoping one of the gurus here can 
determine whether this is a bug or if I'm just doing something annoying.


Any insight that can be provided would be much appreciated.

Regards,

Jordan

Re: APU4 hardware network interfaces tied together

[Jordan Geoghegan]

On 11/15/20 12:25 PM, Mihai Popescu wrote:

Hello,

In the scenario of building a router with APU4, one interface is for wan,
the rest of three are free to use.
What is the most sane and performance wise ( CPU load, interface load,
etc.) way to tie together the remaining three interfaces as a switch, and
avoid using one IP class per interface?
Is it better to use one for lan, leave the remaining two unused and cascade
a dumb switch for other lan connections?

Thank you.


I wouldn't recommend putting the remaining ports into a bridge 
configuration as that will force the interfaces into promiscuous mode, 
and cause higher CPU load. It would be better to just run the LAN off of 
a switch connected to a single port on the APU as that will allow LAN 
traffic to flow without the APU having to touch every single packet. If 
you wanted to be pedantic, an argument could also be made that using a 
single interface would also lend itself to maximally effective interrupt 
coalescing.


Regards,

Jordan

Re: OBSD 6.8 vlan communication issues

[Jordan Geoghegan]

On 11/11/20 3:06 PM, len zaifman wrote:
I am setting up a new system as a firewall using OpenBSD 6.8 current 
-uname -a

OpenBSD fw1.lfz.net 6.8 GENERIC.MP#175 amd64.

I have 3 vlans 70,77,79 on  the firewall using two em devices, em0 and 
em1, in an aggregation to serve these vlans.



There is a Unifi switch which has 2 ports (where em0,em1 are attached) 
set up to pass tagged vlans 70,77,79. The switch ip is 10.10.70.3.


I have a linux host setup on vans 70,77,79 and at address 77 - 
10.10.70.77, 10.10.77.77,10.10.79.77.



So far i cannot communicate over the vlans. Before I vlanned these 
subnets : ie only vlan 1 everywhere - communication worked fine.


So i do not believe there is a physical issue. The issues arose with 
the introduction of the vlans. Is there a configuration issue that 
anyone can spot?



Thank you for any help you can give.

Evidence:

ping on the firewall works locally

for n in 0 7 9 ; do ping -c 2 10.10.7${n}.1 ; done
PING 10.10.70.1 (10.10.70.1): 56 data bytes
64 bytes from 10.10.70.1: icmp_seq=0 ttl=255 time=0.037 ms
64 bytes from 10.10.70.1: icmp_seq=1 ttl=255 time=0.025 ms

--- 10.10.70.1 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 0.025/0.031/0.037/0.006 ms
PING 10.10.77.1 (10.10.77.1): 56 data bytes
64 bytes from 10.10.77.1: icmp_seq=0 ttl=255 time=0.038 ms
64 bytes from 10.10.77.1: icmp_seq=1 ttl=255 time=0.025 ms

--- 10.10.77.1 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 0.025/0.031/0.038/0.006 ms
PING 10.10.79.1 (10.10.79.1): 56 data bytes
64 bytes from 10.10.79.1: icmp_seq=0 ttl=255 time=0.038 ms
64 bytes from 10.10.79.1: icmp_seq=1 ttl=255 time=0.025 ms

--- 10.10.79.1 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 0.025/0.032/0.038/0.007 ms


ping to the switch does not work

ping -c 2 10.10.70.3
PING 10.10.70.3 (10.10.70.3): 56 data bytes

--- 10.10.70.3 ping statistics ---
2 packets transmitted, 0 packets received, 100.0% packet loss

ping to the linux host does not work.

ping -c 2 10.10.70.3
PING 10.10.70.3 (10.10.70.3): 56 data bytes

--- 10.10.70.3 ping statistics ---
2 packets transmitted, 0 packets received, 100.0% packet loss
[13:47:04] leonardz@fw1 etc>>for n in 0 7 9 ; do ping -c 2 
10.10.7${n}.77 ; done

PING 10.10.70.77 (10.10.70.77): 56 data bytes

--- 10.10.70.77 ping statistics ---
2 packets transmitted, 0 packets received, 100.0% packet loss
PING 10.10.77.77 (10.10.77.77): 56 data bytes

--- 10.10.77.77 ping statistics ---
2 packets transmitted, 0 packets received, 100.0% packet loss
PING 10.10.79.77 (10.10.79.77): 56 data bytes

--- 10.10.79.77 ping statistics ---
2 packets transmitted, 0 packets received, 100.0% packet loss

I did the tests both with pfctl -e (enabled) and pfctl -d (disabled). 
It made no difference



The setup is described below

Here is the setup:

= hostname.aggr0
debug
trunkport em0
trunkport em1
up
inet 10.10.70.1/24
alias  10.10.77.1/24
alias  10.10.79.1/24


= hostname.em0
up

= hostname.em1
up


= hostname.vlan70
parent aggr0 vnetid 70
10.10.70.0/24

= hostname.vlan77
parent aggr0 vnetid 77
10.10.77.0/24

= hostname.vlan79
parent aggr0 vnetid 79
10.10.79.0/24


Ifconfig -A shows the vlans are setup

= aggr0
aggr0: flags=8847 mtu 1500
    lladdr fe:e1:ba:d0:f4:8c
    index 6 priority 0 llprio 7
    trunk: trunkproto lacp
    trunk id: [(8000,fe:e1:ba:d0:f4:8c,0006,,),
         (8000,e0:63:da:8e:78:d7,03E8,,)]
        em0 lacp actor system pri 0x8000 mac fe:e1:ba:d0:f4:8c, key 
0x6, port pri 0x8000 number 0x1
        em0 lacp actor state 
activity,aggregation,sync,collecting,distributing
        em0 lacp partner system pri 0x8000 mac e0:63:da:8e:78:d7, key 
0x3e8, port pri 0x1 number 0x9
        em0 lacp partner state 
activity,aggregation,sync,collecting,distributing

        em0 port active,collecting,distributing
        em1 lacp actor system pri 0x8000 mac fe:e1:ba:d0:f4:8c, key 
0x6, port pri 0x8000 number 0x2
        em1 lacp actor state 
activity,aggregation,sync,collecting,distributing
        em1 lacp partner system pri 0x8000 mac e0:63:da:8e:78:d7, key 
0x3e8, port pri 0x1 number 0xa
        em1 lacp partner state 
activity,aggregation,sync,collecting,distributing

        em1 port active,collecting,distributing
    groups: aggr
    media: Ethernet autoselect
    status: active
    inet 10.10.70.1 netmask 0xff00 broadcast 10.10.70.255
    inet 10.10.77.1 netmask 0xff00 broadcast 10.10.77.255
    inet 10.10.79.1 netmask 0xff00 broadcast 10.10.79.255

= em0
em0: flags=8843 mtu 1500
    lladdr fe:e1:ba:d0:f4:8c
    index 1 priority 0 llprio 3
    trunk: trunkdev aggr0
    media: Ethernet autoselect (1000baseT full-duplex)
    status: active

= em1
em1: flags=8843 mtu 1500
    lladdr fe:e1:ba:d0:f4:8c
    index 

Re: home printer

[Jordan Geoghegan]

On 2020-09-17 12:15, Greg Thomas wrote:

I've always been happy with the cheap Brother laser printers with ethernet,
even with just their version of Postscript.  But I believe they still sell
Postscript printers, too.

On Thu, Sep 17, 2020 at 10:07 AM Ingo Schwarze  wrote:


Hi Carson,

Carson Chittom wrote on Thu, Sep 17, 2020 at 09:51:45AM -0500:

Jan Stary  writes:

Can people please recommend a home laser printer
that is known to work well with OpenBSD?

I would like to avoid cups, and possibly a2ps
and foo* and if= and all that dance
- a printer that speaks postscript and is as easy as
lp:lp=/dev/lp:sd=/var/spool/output/lpd:lf=/var/log/lpd-errs:

HP at least used to (and I assume still do) make several decent
printers that spoke Postscript.

That answer used to be spot on until about the year 2000.  After
that, quality of HP laser printers went down the drain very rapidly.
One office i worked in decided in 2003 that the then more then five
year old HP LaserJet might die from old age soon and bought a new
one to be safe and not experience service disruption.  The old one
was left running, too, because why not, and printing traffic was
shared about evenly between the two because people tended to use
the one closest to their desk.

When the *successor* of the new one died from old age about six to
eight years later (i.e. when two of the new ones had worn out one
after the other, don't remember how long they lasted exactly, but
not longer than three or four years i think), the old one was still
going strong.  If i remember correctly, when the pre-2000 one finally
did die from old age, it was probably fifteen years old, if not
more, with continuous office use.

I doubt HP printers have become better again, but i'm not sure.


In particular, I've used the
CP1525nw in the past with OpenBSD.  Haven't tried it in a couple
years, though; none of my OpenBSD machines need to print, these
days.

Same here.  Currently, a Kyocera P2135dn is sitting on the desk here,
but i can't say whether it is good because i'm printing so little.

To the OP, what matters is a decent PostScript Processor
and a RJ45 Ethernet connector, then it will work with OpenBSD
no matter what.

Yours,
   Ingo




Greg has the right idea. I've had a great experience with my Brother 
MFC-L5700 laser printer. I can print Postscript and PDF documents to it 
directly using lpd/lpr in the OpenBSD base system, and the printer 
supports directly uploading scans to an FTP/SFTP server. It's quite nice 
being able to print and scan without messing around with drivers/cups 
and the dark arts magic required to make it work reliably.


I set my printer up by enabling lpd and adding a single line in my 
/etc/printcap file:


lp|remote line printer:\
:lp=:rm=192.0.2.5:rp=lp:sd=/var/spool/output/lpd:lf=/var/log/lpd-errs:



So far this setup has been totally bulletproof and has yet to fail me 
after over 5000 scanned pages and a couple thousand print jobs.


Regards,

Jordan

Re: Very slow clock in Debian vmm guest

[Jordan Geoghegan]

If you check the mailing list archives, you will see that this issue has 
been discussed extensively.


Dave Voutila has written a linux vmm kernel driver to work around some 
of the issues:


https://github.com/voutilad/virtio_vmmci

Regards,

Jordan

On 2020-08-28 20:48, Aaron Miller wrote:

I have a debian testing guest running in vmm(4) on my -current
system, and the internal clock is very slow. For example running
`sleep 3` takes about 10 seconds of real time to run. This is too
much for ntpd to correct, unfortunately.

Anyone know what the problem is and how I might go about fixing
it? Thanks!

--Aaron

OpenBSD 6.7-current (GENERIC.MP) #36: Sat Aug 22 11:27:03 MDT 2020
 dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENE
RIC.MP
real mem = 16827916288 (16048MB)
avail mem = 16302870528 (15547MB)
random: good seed from bootblocks
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xccbfd000 (65 entries)
bios0: vendor LENOVO version "N14ET37W (1.15 )" date 09/06/2016
bios0: LENOVO 20BSCTO1WW
acpi0 at bios0: ACPI 5.0
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP SLIC ASF! HPET ECDT APIC MCFG SSDT SSDT
SSDT SSDT SSDT SSDT SSDT SSDT SSDT SSDT PCCT SSDT TCPA SSDT UEFI
MSDM BATB FPDT UEFI DMAR
acpi0: wakeup devices LID_(S4) SLPB(S3) IGBE(S4) EXP2(S4) XHCI(S3)
EHC1(S3)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpihpet0 at acpi0: 14318179 Hz
acpiec0 at acpi0
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM) i7-5600U CPU @ 2.60GHz, 798.30 MHz, 06-3d-
04
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,
PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,D
TES64,MWAIT,DS-
CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE
4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAG
E1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI
1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,RDSEED,ADX,SMAP,PT,MD_CLEAR,
IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4.1.1.1, IBE
cpu1 at mainbus0: apid 1 (application processor)
cpu1: Intel(R) Core(TM) i7-5600U CPU @ 2.60GHz, 798.16 MHz, 06-3d-
04
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,
PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,D
TES64,MWAIT,DS-
CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE
4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAG
E1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI
1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,RDSEED,ADX,SMAP,PT,MD_CLEAR,
IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 1, core 0, package 0
cpu2 at mainbus0: apid 2 (application processor)
cpu2: Intel(R) Core(TM) i7-5600U CPU @ 2.60GHz, 798.17 MHz, 06-3d-
04
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,
PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,D
TES64,MWAIT,DS-
CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE
4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAG
E1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI
1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,RDSEED,ADX,SMAP,PT,MD_CLEAR,
IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 1, package 0
cpu3 at mainbus0: apid 3 (application processor)
cpu3: Intel(R) Core(TM) i7-5600U CPU @ 2.60GHz, 798.16 MHz, 06-3d-
04
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,
PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,D
TES64,MWAIT,DS-
CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE
4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAG
E1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI
1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,RDSEED,ADX,SMAP,PT,MD_CLEAR,
IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 1, core 1, package 0
ioapic0 at mainbus0: apid 2 pa 0xfec0, version 20, 40 pins
acpimcfg0 at acpi0
acpimcfg0: addr 0xf800, bus 0-63
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus -1 (PEG_)
acpiprt2 at acpi0: bus 3 (EXP1)
acpiprt3 at acpi0: bus 4 (EXP2)
acpiprt4 at acpi0: bus -1 (EXP3)
acpiprt5 at acpi0: bus -1 (EXP6)
acpicpu0 at acpi0: C3(200@233 mwait.1@0x40), C2(200@148
mwait.1@0x33), C1(1000@1 mwait.1), PSS
acpicpu1 at acpi0: C3(200@233 mwait.1@0x40), C2(200@148
mwait.1@0x33), C1(1000@1 mwait.1), PSS
acpicpu2 at acpi0: C3(200@233 mwait.1@0x40), C2(200@148
mwait.1@0x33), C1(1000@1 mwait.1), PSS
acpicpu3 at acpi0: C3(200@233 mwait.1@0x40), C2(200@148
mwait.1@0x33), C1(1000@1 

Re: Installation Media Self Integrity Check

[Jordan Geoghegan]

On 2020-08-12 21:30, Dan Peretz wrote:

Hello, the FAQ states this:
"The installXX.iso and installXX.fs images do not contain an
SHA256.sig file, so the installer will complain that it can't check
the signature of the included sets [...] This is because it would make
no sense for the installer to verify them. If someone were to make a
rogue installation image, they could certainly change the installer to
say the files were legitimate."
Although that's true for intentional modifications, it would still be
useful to have the installation medium perform a self integrity check
against accidental or natural data corruption. For example, Ubuntu
recently enabled a by-default integrity check, starting with release
20.04:
"Ubuntu now defaults to checking the integrity of the medium in use
when booting into live sessions. This can be skipped by hitting
Ctrl-C. We’ve enabled this because failed installs due to corrupt
downloads of installation media is one of the most common error
conditions that users encounter." (Source:
)
I would like to have OpenBSD include at least an unsigned SHA256 file
in the discs. The installer would then detect that the checksums are
unsigned and warn about the security implications, but it would let
the user run the check. I think it would be wise to make it check the
bsd.rd image that's actually booted when booting from the disc, and
not just the bsd.rd file set. (I get that the OpenBSD installer is
just a multipurpose "bsd.rd" RAM disk that can be found not just in
the installation discs, right?)

Thank you!



I think the whole idea is to check the ISO itself with signify before 
you even boot/write it. If you're concerned about whether the ISO was 
written to your install media correctly (or its integrity after the 
fact), then it should be pretty straightforward to check if the hashes 
match up. Once you've verified the SHA256 file from the mirrors, you 
know the hashes are valid (and if you're worried about MITM/compromised 
web hosts, the signify keys are published all over social media and 
blogs etc). Signify does its best to cope with the chicken and the egg 
problem (and everything that comes with trusting trust), but at the end 
of the day, you have to draw the line somewhere.


If you're spooked about the SHA256.sig warning, then you can install 
over the network from a mirror, but ultimately it's irrelevant. Check 
your ISO with signify and you're good to go.

Re: Adding more syspatch platform.

[Jordan Geoghegan]

On 2020-08-13 02:39, Sebastian Benoit wrote:

Jordan Geoghegan(jor...@geoghegan.ca) on 2020.08.12 10:32:21 -0700:


On 2020-08-12 02:08, Stuart Henderson wrote:

The only proxy we have for "what is really used" is dmesg submissions.
Since 6.7 release:

amd64   62
i3865
arm64   3
macppc  2
octeon  1

Based on this there isn't a great case for adding any more.


I didn't realize you guys used dmesg@ as a popularity gauge, I thought

s/popularity/usage/


it was just for sending dmesgs for interesting/new hardware. I figured
with something like Edgerouters with their standardized hardware that
repeat dmesgs would just serve to irritate the devs. I personally
administer more OpenBSD 6.x machines than are on that list you sent. I
can start hammering dmesg@, but then I'm gonna skew your stats and

In a way, you also skew the stats if you dont send dmesgs. And "you" means
everyone. Do it once every 6 months (for every release) i'd say.


you're gonna think that your userbase consists of a bunch of autists
that unironically run macppc, sparc64 and octeon in production
everywhere. In the small Canadian town I live in, I've got a big chunk
of it running on OpenBSD. I've got a bunch of old 90's / 2000's i386
stuff too, so I can send dmesgs in for that too, the reason I didn't is
I figured the hardware was already 'been there, done that'.

If the dmesgs dont show that certain hardware is still used, the drivers for
it will get deleted eventually. Another reason to show what you use.


As I mentioned earlier in the thread, I can afford to donate 2 octeon
machines to any devs that are interested (including shipping world wide;
any devs reading: please contact me if you're interested), and am also
happy to help out with octeon stuff in any way I can. Obviously you guys
aren't going to trust me to do anything syspatch related as I'm not a
dev, but I'd at least like it to be known that there are people who care
about the octeon port and who are willing make an effort for it.

Regards,

Jordan




Okay, thanks for the info Sebastian, I'll start doing twice annual dumps 
to dmesg@ for all my machines.


Regards,

Jordan

Re: Adding more syspatch platform.

[Jordan Geoghegan]

On 2020-08-12 02:08, Stuart Henderson wrote:

The only proxy we have for "what is really used" is dmesg submissions.
Since 6.7 release:

amd64   62
i3865
arm64   3
macppc  2
octeon  1

Based on this there isn't a great case for adding any more.



I didn't realize you guys used dmesg@ as a popularity gauge, I thought 
it was just for sending dmesgs for interesting/new hardware. I figured 
with something like Edgerouters with their standardized hardware that 
repeat dmesgs would just serve to irritate the devs. I personally 
administer more OpenBSD 6.x machines than are on that list you sent. I 
can start hammering dmesg@, but then I'm gonna skew your stats and 
you're gonna think that your userbase consists of a bunch of autists 
that unironically run macppc, sparc64 and octeon in production 
everywhere. In the small Canadian town I live in, I've got a big chunk 
of it running on OpenBSD. I've got a bunch of old 90's / 2000's i386 
stuff too, so I can send dmesgs in for that too, the reason I didn't is 
I figured the hardware was already 'been there, done that'.


As I mentioned earlier in the thread, I can afford to donate 2 octeon 
machines to any devs that are interested (including shipping world wide; 
any devs reading: please contact me if you're interested), and am also 
happy to help out with octeon stuff in any way I can. Obviously you guys 
aren't going to trust me to do anything syspatch related as I'm not a 
dev, but I'd at least like it to be known that there are people who care 
about the octeon port and who are willing make an effort for it.


Regards,

Jordan

Re: How many IPs can I block before taking a performance hit?

[Jordan Geoghegan]

On 2020-08-12 05:11, Alan McKay wrote:

Hey folks,

This is one that is difficult to test in a test environment.

I've got OpenBSD 6.5 on a relatively new pair of servers each with 8G RAM.

With some scripting I'm looking at feeding block IPs to the firewalls
to block bad-guys in near real time, but in theory if we got attacked
by a bot net or something like that, it could result in a few thousand
IPs being blocked.  Possibly even 10s of thousands.

Are there any real-world data out there on how big of a block list we
can handle without impacting performance?

We're doing the standard /etc/blacklist to load a table and then have
a block on the table right at the top of the ruleset.

thanks,
-Alan




At Otto said, if you're using tables, then you should be fine. I'm doing 
geoip blocking and all sorts of filtering using a pf table that contains 
over 200 undecillion addresses (that obviously includes CIDR block 
expansion):


# Entries (+-)
9482 addresses added.
10859 addresses deleted.

# Entries (expanded CIDR blocks)
IPv4 addresses in table:  966545967
IPv6 addresses in table:  298179424470603435988810818668701155328

fw$ wc -l < /etc/pf-badhost.txt
  146541

Re: Adding more syspatch platform.

[Jordan Geoghegan]

On 2020-08-11 20:35, Theo de Raadt wrote:

Jordan Geoghegan  wrote:


On 2020-08-11 15:50, Theo de Raadt wrote:

Predrag Punosevac  wrote:


Theo de Raadt  wrote:


No, it is a question of which additional platform, you avoided that
didn't you


octeon is the only one I can think of.

read below:


if you name one that less than 100 people use, then well come on

Not that it matters, but I have well over 100 users of OpenBSD on
octeon for the different client sites I manage. I'd love to see
syspatches for octeon, but I have no business telling you guys what to
do.

The people involved in building the syspatches don't even have octeons.



I'd be happy to donate a couple of octeon machines if access to hardware 
is an issue. I'm just a random internet person, so I imagine you guys 
wouldn't trust me to do the work to build the syspatches, but please do 
let me know if some octeon machines would be useful to you guys. I'd 
happily do the legwork myself but there doesn't appear to be a way to 
manually syspatch (please correct me if I'm wrong).


Regards,

Jordan

Re: Adding more syspatch platform.

[Jordan Geoghegan]

On 2020-08-11 15:50, Theo de Raadt wrote:

Predrag Punosevac  wrote:


Theo de Raadt  wrote:


No, it is a question of which additional platform, you avoided that
didn't you


octeon is the only one I can think of.

read below:


if you name one that less than 100 people use, then well come on


Not that it matters, but I have well over 100 users of OpenBSD on octeon 
for the different client sites I manage. I'd love to see syspatches for 
octeon, but I have no business telling you guys what to do.


Regards,

Jordan

Re: Way to find most active IPs for rate limiting with pf

[Jordan Geoghegan]

On 2020-08-06 13:46, Alan McKay wrote:

So I want to implement rate limiting, and to determine a reasonable
rate based on current traffic patterns I'd like to be able to figure
out which source IPs are generating the most connections and at what
rate.

Is there a way to do that?




There is likely a better way to do it, but a quick and dirty method to 
check for number of states for an IP address would be something like:


 $  doas pfctl -s state | grep -Fc "192.0.2.4"

or if you wanted to check for the number of tcp or udp states specifically:

  $  doas pfctl -s state | grep "^all tcp" | grep -Fc "192.0.2.4"


Regards,

Jordan

Re: CPU usage of httpd+slowcgi

[Jordan Geoghegan]

On 2020-07-24 03:16, Kihaguru Gathura wrote:

Hi,

Which of the following legacy CPU types is best suited for very busy web
server httpd+slowcgi

Niagara CPU Such as T2 - More parallel Threads and Low power per single
thread
Sparc64 CPU such as VI, VII - Fewer threads but more computing power per
thread.

How is multithreading utilization of httpd+slowcgi like?

Kind regards,

Kihaguru.


Hi  Kihaguru,

As with any computer, newer tends to be better with Moore's Law and all 
that. On sparc64 most of the logical cores that are shown are really 
just SMT pretending to be a bunch of cores. I have one machine that 
claims 128 cores, but in reality, its just 16 cores with 8-way SMT. 
sparc64 isn't renowned for its single core execution speed, so the 
faster the better in that regard.


In my experience with running OpenBSD on sparc64, the kernel biglock or 
crypto became a bottleneck before other things did. (I've used T3 and T4 
machines fairly extensively with OpenBSD). I've found that disk 
activity, networking and/or TLS would bottleneck before httpd became a 
bottleneck when I was running sparc64 web servers in production. If you 
are running very heavy scripts/programs with slowcgi, then you're 
results may be different.


Things have likely improved dramatically in the past year or two with 
all the work done on removing the biglock, but the moral of the story 
remains, fewer, faster cores are likely to produce superior performance 
to numerous low power cores.


Regards,

Jordan

Re: video capture / streaming

[Jordan Geoghegan]

On 2020-07-24 08:06, Rudolf Sykora wrote:

Dear list,


I'd like to stream video from my (usb-connected) camera via a web
server. The 1st I tried was to see if the camera works. So:

odin$ video
video: /dev/video: Permission denied
odin$ doas video
No protocol specified
video: cannot open display :0.0
odin$ xhost +
access control disabled, clients can connect from any host
odin$ doas video
^Codin$

Ie, at last I saw an X window with the video. But is it possible to run
the video command as an ordinary user? (I had to doas...)

Next I want to stream the video via some kind of a server.
On the internet I saw people using 'ffmpeg' to serve the file somehow,
as well as using 'nginx' with the 'rtmp' module. Is any of this the way
how you would achieve the goal, or is it possible to use, say, the 'httpd'
server?

At this moment I know very little both about streaming and available
servers to do the job, and that's why I am asking for recommendation
based on some experience and knowledge.

Thank you for any comments.


Ruda


Hi Ruda,

You can configure FFmpeg to dump an HLS stream into a location served by 
httpd. FFmpeg does support some USB cameras using the V4L2 protocol, but 
you may need to play around with formats and resolutions to get the 
optimal experience/performance. I've done this extensively, as I'm 
currently working on a CCTV project that will run natively on OpenBSD.


Regards,

Jordan

Re: Potential grep bug?

[Jordan Geoghegan]

Hi Demi,

On 2020-07-10 22:42, Demi M. Obenour wrote:

On 2020-06-23 22:29, Jordan Geoghegan wrote:

Hello,

I was working on a couple POSIX regular expressions to search for and validate 
IPv4 and IPv6 addresses with optional CIDR blocks, and encountered some strange 
behaviour from the base system grep.

I wanted to validate my regex against a list of every valid IPv4 address, so I 
generated a list with a zsh 1 liner:

  for i in {0..255}; do; echo $i.{0..255}.{0..255}.{0..255} ; done | tr 
'[:space:]' '\n' > IPv4.txt

My intentions were to test the regex by running it with 'grep -c' to confirm 
there was indeed 2^32 addresses matched, and I also wanted to benchmark and 
compare performance between BSD grep, GNU grep and ripgrep. The command I used:

    grep -Eoc 
"((25[0-5]|(2[0-4]|1{0,1}[[:digit:]]){0,1}[[:digit:]])\.){3,3}(25[0-5]|(2[0-4]|1{0,1}[[:digit:]]){0,1}[[:digit:]])(/[1-9]|/[1-2][[:digit:]]|/3[0-2])?"

My findings were surprising. Both GNU grep and ripgrep were able get through 
the file in roughly 10 and 20 minutes respectively, whereas the base system 
grep took over 20 hours! What interested me the most was that the base system 
grep when run with '-c' returned '0' for match count. It seems that 'grep -c' 
will have its counter overflow if there are more than 2^32-1 matches 
(4294967295) and then the counter will start counting from zero again for 
further matches.

Does OpenBSD use an NFA/DFA regular expression implementation, or a
backtracking one?  If it uses the latter, then your regex is probably
causing catastrophic backtracking.


Regards,

Jordan

Sincerely,

Demi



Ya you're probably right, I know GNU grep does a whole bunch of fancy 
optimizations that aren't done in BSD grep, the old GNU grep maintainer 
did a small write up on the FreeBSD mailing list about some of their tricks:

https://lists.freebsd.org/pipermail/freebsd-current/2010-August/019310.html

My IPv6 regex causes grep to randomly segfault on MacOS (which uses an 
older version of the BSD/FreeGrep that OpenBSD uses I believe), so I 
imagine there's something going on under the hood with these regexes 
making grep thrash hard.



Regards,

Jordan

PF-BadHost Patch

[Jordan Geoghegan]

Hi everybody,

I don't want to spam the list here, but I figured this was important 
enough to warrant an announcement considering several hundred people 
downloaded the script last week.


tl;dr: Bugs found, patch your stuff

A couple bugs have been reported by users, one in RipGrep and another in 
find(1).
The RipGrep bug is a bit of a show-stopper, as it was discovered that 
its behaviour differed from that of grep(1) and GNU grep, where with the 
'-o' flag, it does not select the most exact match when given multiple 
search patterns ala 'pattern1|pattern2' it rather will select the first 
specified pattern that happens to match. What makes this significant is 
that the way this played out was RipGrep was only pulling the first 
digit in the CIDR block, ie '192.0.2.0/24' was instead matched as 
'192.0.2.0/2' . Big yikes. Mea culpa, should of caught that before I 
tried to be a tough guy and recommend an alternative to tried and true 
grep.


The find(1) bug has been quite persnickety to debug, as users report 
intermittent failures from find(1) where it pukes up error: 'find: 
-exec: cannot open "." '
I haven't been able to reproduce the error, so until I (or someone more 
knowledgeable) can figure out what's going on with that, that's going to 
have to be worked around.


I've released a patch to address both issues, you can find full 
instructions on the website: geoghegan.ca/pfbadhost.html


Quick start:

$ ftp https://geoghegan.ca/pub/pf-badhost/0.4/patches/pf-badhost_p0.patch
# patch Hey folks, just thought I'd share with you that I've released the 
latest versions of pf-badhost and unbound-adblock.


pf-badhost webpage: https://www.geoghegan.ca/pfbadhost.html
unbound-adblock webage: https://www.geoghegan.ca/unbound-adblock.html

Key pf-badhost changes:

* pf-badhost goes portable, we now support 
{Open,Free,Net,Dragonfly}BSD as well as MacOS!
* Support for IPv6 subnet aggregation added thanks to the excellent 
aggregate6 utility written by job@

* Greatly improved IPv6 handling in general
* User configuration section added for configuring whitelists and 
custom blocklists

* Bogon filtering added
* Greatly improved error handling


Key unbound-adblock changes:

* unbound-adblock goes portable, we now support 
{Open,Free,Net,Dragonfly}BSD as well as Linux!

* Greatly improved error handling and input sanitation
* User configuration section added for configuring whitelists and 
custom blocklists


pf-badhost changelog: 
https://www.geoghegan.ca/pub/pf-badhost/0.4/changelog.txt
unbound-adblock changelog: 
https://www.geoghegan.ca/pub/unbound-adblock/0.4/changelog.txt

Re: sysupgrade failure due to boot.conf

[Jordan Geoghegan]

On 2020-07-10 15:37, Alfred Morgan wrote:

Please, I have had this problem for several versions now and it still isn't
working right.
I have this on all three of my servers:
echo boot > /etc/boot.conf

I have this boot.conf because openbsd fails to boot (on all three servers)
because it hangs on the boot> prompt because of some ghost input (I have no
keyboard plugged in). So I was told that putting "boot" in my boot.conf
would solve the problem and it did the trick. BUT, sysupgrade now fails
trying to upgrade 6.6 -> 6.7. So what can I put in my boot.conf that will
ignore the ghost input in the boot> prompt and allow sysupgrade to succeed?

I feel that it's a bug in sysupgrade that it doesn't behave the same having
"boot" in the boot.conf. Any help?

-alfred



Relevant xkcd? https://xkcd.com/1172/

Re: Unbound Configuration

[Jordan Geoghegan]

On 2020-07-10 14:29, ken.hendrick...@l3harris.com wrote:

--- I asked:

What I would like to do now is make the *simplest
possible* unbound.conf file and get it working.

Thinking that an absolutely empty unbound.conf file
would be the simplest, I tried it.  It doesn't work.

Can anybody help me with the simplest possible
unbound.conf file???

Thanks,
Ken


   


CONFIDENTIALITY NOTICE: This email and any attachments are for the sole use of 
the intended recipient and may contain material that is proprietary, 
confidential, privileged or otherwise legally protected or restricted under 
applicable government laws. Any review, disclosure, distributing or other use 
without expressed permission of the sender is strictly prohibited. If you are 
not the intended recipient, please contact the sender and delete all copies 
without reading, printing, or saving.




This FAQ page should be a useful starting point:

https://www.openbsd.org/faq/pf/example1.html#dns

pf-badhost + unbound adblock v4 released

[Jordan Geoghegan]

Hey folks, just thought I'd share with you that I've released the latest 
versions of pf-badhost and unbound-adblock.


pf-badhost webpage: https://www.geoghegan.ca/pfbadhost.html
unbound-adblock webage: https://www.geoghegan.ca/unbound-adblock.html

Key pf-badhost changes:

* pf-badhost goes portable, we now support {Open,Free,Net,Dragonfly}BSD 
as well as MacOS!
* Support for IPv6 subnet aggregation added thanks to the excellent 
aggregate6 utility written by job@

* Greatly improved IPv6 handling in general
* User configuration section added for configuring whitelists and custom 
blocklists

* Bogon filtering added
* Greatly improved error handling


Key unbound-adblock changes:

* unbound-adblock goes portable, we now support 
{Open,Free,Net,Dragonfly}BSD as well as Linux!

* Greatly improved error handling and input sanitation
* User configuration section added for configuring whitelists and custom 
blocklists


pf-badhost changelog: 
https://www.geoghegan.ca/pub/pf-badhost/0.4/changelog.txt
unbound-adblock changelog: 
https://www.geoghegan.ca/pub/unbound-adblock/0.4/changelog.txt

Re: How do I get the man page for a package I haven't installed yet?

[Jordan Geoghegan]

On 2020-06-26 20:03, Theo de Raadt wrote:

Jordan Geoghegan  wrote:


On 2020-06-26 18:45, Theo de Raadt wrote:

Jordan Geoghegan  wrote:



On 2020-06-26 13:43, Marc Espie wrote:

On Tue, Jun 23, 2020 at 12:20:35PM -0600, Theo de Raadt wrote:

Ottavio Caruso  wrote:


Hi,

Unless I've got it all wrong, <https://man.openbsd.org/> will only
display man pages for programs and commands in base. Is there a way to
display the man page for a package/port I haven't installed and/or
downloaded yet? (This assumes I haven't downloaded the ports cvs
tree).

Doing that would be very annoying and painful, and very few people
would want it.  It would also substantially degrade the clarity at
man.openbsd.org

Actually, it ought to be feasible to have the same mechanism in place for
base  as a third party mechanism.

I don't think it would be that difficult to setup, this obviously ought to
be separate from the main OpenBSD installation, as the quality of manpages
from ports is often not up-to-par compared to base.

Both Ingo and naddy and I, we've been routinely passing all manpages from
all packages through groff and mandoc and makewhatis to the point that
over 99% of them would be clean for a usage similar to man.openbsd.org


FreeBSD appears to offer manual pages from ports on their man page
website: https://www.freebsd.org/cgi/man.cgi

Not advocating for anything, just thought I'd point it out.

Completely irrelevant.


I thought it was relevant for folks looking for http access to ports
manpages, as the FreeBSD and OpenBSD ports trees overlap
significantly. I often use that site when I'm on a machine that
doesn't happen to have the particular package installed whose manpage
I want to view.

It is very easy for outsiders to ask a project to do more, MORE MORE
MORE, and not understand there are a limited number of people doing the
work.

So if this gets done, something else will not get done, or will get done
less well.

And it will be your fault.




I wasn't asking for anything, I was just trying to be helpful and share 
a resource I've personally found useful. I don't feel strongly about any 
of this, so consider the conversation over.


Regards,

Jordan

Re: How do I get the man page for a package I haven't installed yet?

[Jordan Geoghegan]

On 2020-06-26 18:45, Theo de Raadt wrote:

Jordan Geoghegan  wrote:




On 2020-06-26 13:43, Marc Espie wrote:

On Tue, Jun 23, 2020 at 12:20:35PM -0600, Theo de Raadt wrote:

Ottavio Caruso  wrote:


Hi,

Unless I've got it all wrong, <https://man.openbsd.org/> will only
display man pages for programs and commands in base. Is there a way to
display the man page for a package/port I haven't installed and/or
downloaded yet? (This assumes I haven't downloaded the ports cvs
tree).

Doing that would be very annoying and painful, and very few people
would want it.  It would also substantially degrade the clarity at
man.openbsd.org

Actually, it ought to be feasible to have the same mechanism in place for
base  as a third party mechanism.

I don't think it would be that difficult to setup, this obviously ought to
be separate from the main OpenBSD installation, as the quality of manpages
from ports is often not up-to-par compared to base.

Both Ingo and naddy and I, we've been routinely passing all manpages from
all packages through groff and mandoc and makewhatis to the point that
over 99% of them would be clean for a usage similar to man.openbsd.org


FreeBSD appears to offer manual pages from ports on their man page
website: https://www.freebsd.org/cgi/man.cgi

Not advocating for anything, just thought I'd point it out.

Completely irrelevant.



I thought it was relevant for folks looking for http access to ports 
manpages, as the FreeBSD and OpenBSD ports trees overlap significantly. 
I often use that site when I'm on a machine that doesn't happen to have 
the particular package installed whose manpage I want to view.


Regards,

Jordan

Re: How do I get the man page for a package I haven't installed yet?

[Jordan Geoghegan]

On 2020-06-26 13:43, Marc Espie wrote:

On Tue, Jun 23, 2020 at 12:20:35PM -0600, Theo de Raadt wrote:

Ottavio Caruso  wrote:


Hi,

Unless I've got it all wrong,  will only
display man pages for programs and commands in base. Is there a way to
display the man page for a package/port I haven't installed and/or
downloaded yet? (This assumes I haven't downloaded the ports cvs
tree).

Doing that would be very annoying and painful, and very few people
would want it.  It would also substantially degrade the clarity at
man.openbsd.org

Actually, it ought to be feasible to have the same mechanism in place for
base  as a third party mechanism.

I don't think it would be that difficult to setup, this obviously ought to
be separate from the main OpenBSD installation, as the quality of manpages
from ports is often not up-to-par compared to base.

Both Ingo and naddy and I, we've been routinely passing all manpages from
all packages through groff and mandoc and makewhatis to the point that
over 99% of them would be clean for a usage similar to man.openbsd.org



FreeBSD appears to offer manual pages from ports on their man page 
website: https://www.freebsd.org/cgi/man.cgi


Not advocating for anything, just thought I'd point it out.

Potential grep bug?

[Jordan Geoghegan]

Hello,

I was working on a couple POSIX regular expressions to search for and 
validate IPv4 and IPv6 addresses with optional CIDR blocks, and 
encountered some strange behaviour from the base system grep.


I wanted to validate my regex against a list of every valid IPv4 
address, so I generated a list with a zsh 1 liner:


 for i in {0..255}; do; echo $i.{0..255}.{0..255}.{0..255} ; done | 
tr '[:space:]' '\n' > IPv4.txt


My intentions were to test the regex by running it with 'grep -c' to 
confirm there was indeed 2^32 addresses matched, and I also wanted to 
benchmark and compare performance between BSD grep, GNU grep and 
ripgrep. The command I used:


   grep -Eoc 
"((25[0-5]|(2[0-4]|1{0,1}[[:digit:]]){0,1}[[:digit:]])\.){3,3}(25[0-5]|(2[0-4]|1{0,1}[[:digit:]]){0,1}[[:digit:]])(/[1-9]|/[1-2][[:digit:]]|/3[0-2])?"


My findings were surprising. Both GNU grep and ripgrep were able get 
through the file in roughly 10 and 20 minutes respectively, whereas the 
base system grep took over 20 hours! What interested me the most was 
that the base system grep when run with '-c' returned '0' for match 
count. It seems that 'grep -c' will have its counter overflow if there 
are more than 2^32-1 matches (4294967295) and then the counter will 
start counting from zero again for further matches.


    ryzen$ time zcat IPv4.txt.gz | grep -Eoc "((25[0-5]|(2[0-4]|1{0,1}...
    0
    1222m09.32s real  1224m28.02s user 1m16.17s system

    ryzen$ time zcat allip.txt.gz | ggrep -Eoc "((25[0-5]|(2[0-4]|1{0,1}...
    4294967296
    10m00.38s real    11m40.57s user 0m30.55s system

    ryzen$ time rg -zoc "((25[0-5]|(2[0-4]|1{0,1}...
    4294967296
    21m06.36s real    27m06.04s user 0m50.08s system

# See the counter overflow/reset:
    jot 4294967350 | grep -c "^[[:digit:]]"
    54

All testing was done on a Ryzen desktop machine running 6.7 stable.

The grep counting bug can be reproduced with this command:
   jot 4294967296 | nice grep -c "^[[:digit:]]"

Regards,

Jordan

Re: Any idea/suggestion for old Cisco router to be use running OpenBSD current for WG?

[Jordan Geoghegan]

I don't know much about Cisco hardware, but I've had great luck with the 
Edgerouter line of products. I've run my home network on an Edgerouter 
Pro for several years now without issue, and have dozens of ER4 and 
ER-Lite devices out in the wild.


If you're looking for non-x86 routing solutions, then the Edgerouter is 
one of the best bets.


Regards,

Jordan

On 2020-06-23 09:01, Daniel Ouellet wrote:

Hi,

This might be a bit weird question, but I saw the wireguard being put in
the kernel in the last few days and I am very existed abut it oppose to
use the package on it and even today there was more on it.

Many thanks for this!!!

I also know there was effort and some Cisco router can run OpenBSD very
well, however I have no clue as to any of this stand now.

I don't have a problem to use APU type or other Ubiquit for small
OpenBSD router, but I wonder about using Cisco instead. The only reason
is for may be more stability, most likely less performance for sure, but
less change to have corrupted reboot on power lost, etc.

And sadly for some customers having what they see as computer as router
don't make them fell good, but seeing a Cisco box kind of wipe out the
impression. I am not saying it's justify, but perception is sometime
everything, but if I have my say in it I want all my routers to be
OpenBSD as much as I can where the needs is not to multiple Gb in speed.

So, any suggestion or updates as to what's now available and hopefully
in use now.

I really don't care for any special model, or even Juniper, as long as I
can put OpenBSD on it.

So any feedback as to where it's stand now and what's usable in a
reliable way would be greatly appreciated.

And yes I know I may well get better performance in some cases with a
small APU device then a Cisco one, but that's for what we all know may
not be logical to be used, but for sadly how some clients may fell, not
knowing any better.

I guess you can see that as some people do security by obstruction, but
we al know it's not more secure, this is routing by obstruction I guess
and may be less performant, but achieve comfort obstruction confidence.

I just have no clue if wireguard needs to be run, what can be achieve as
the CPU in all Cisco device is always under power, we all know that.

This may not go anywhere, however I liked to look even if for nothing
else then just being fun to do if that can't even be usable.

Many thanks for your time and feedback.

Daniel

PS; And yes, that's most likely stupid I know. Sometime what's used is
not always what make sense for other reason that are stupid.

Re: Potential awk bug?

[Jordan Geoghegan]

On 2020-06-13 05:14, Stuart Henderson wrote:

On 2020-06-12, Jordan Geoghegan  wrote:


On 2020-06-08 11:29, Todd C. Miller wrote:

On Sun, 07 Jun 2020 17:02:03 -0700, Jordan Geoghegan wrote:


Thanks for the quick response. I certainly wasn't expecting to find an
ancient bug like this. Should I be reporting this bug upstream, or are
you planning on upstreaming a diff?

I've created a pull request to fix this upstream:
  https://github.com/onetrueawk/awk/pull/80

   - todd

Sorry for the late reply, I've been AFK for the last week.

Excited to see base system awk get synced with upstream. Kudos to you
and Philip for such a swift and effective response.

btw, note that it's not a direct copy of upstream, OpenBSD's version has
various changes (at least pledge and use of bounded string functions).




Thanks for the info Stuart,  that's awesome, I didn't realise awk was 
pledged. I appreciate the knowledge tidbits you drop from time to time. 
Your bit the other day about having ftp(1) output to stdout to drop all 
filesystem privileges blew my mind and I've subsequently updated most of 
my scripts to take advantage of it.


Regards,

Jordan Geoghegan

Re: Potential awk bug?

[Jordan Geoghegan]

On 2020-06-08 11:29, Todd C. Miller wrote:

On Sun, 07 Jun 2020 17:02:03 -0700, Jordan Geoghegan wrote:


Thanks for the quick response. I certainly wasn't expecting to find an
ancient bug like this. Should I be reporting this bug upstream, or are
you planning on upstreaming a diff?

I've created a pull request to fix this upstream:
 https://github.com/onetrueawk/awk/pull/80

  - todd


Sorry for the late reply, I've been AFK for the last week.

Excited to see base system awk get synced with upstream. Kudos to you 
and Philip for such a swift and effective response.


Regards,

Jordan Geoghegan

Re: OpenBSD alternatives to Pi-Hole

[Jordan Geoghegan]

On 2020-06-12 14:01, George wrote:


On 2020-06-12 3:41 p.m., Maurice McCarthy wrote:
You could have a look at 
https://www.geoghegan.ca/unbound-adblock.html and

https://www.geoghegan.ca/pfbadhost.html


Simply great! Will definitely try these out.

Merci!

George



Hey there,

I'm the author of those scripts. In response to concerns about 
heavyness/memory use of DNS blocklists:  unbound-adblock is pretty light 
on memory (~30MB of RAM usage) as we serve NXDOMAIN responses instead of 
redirecting to 0.0.0.0 etc. By doing this we save a massive amount of 
memory that would otherwise be spent mapping each domain to a black hole 
address. I run unbound-adblock on many Edgerouter Lites and havent had 
any issues.


Regards,

Jordan Geoghegan

Re: Potential awk bug?

[Jordan Geoghegan]

Hi Philip,

Thanks for the quick response. I certainly wasn't expecting to find an 
ancient bug like this. Should I be reporting this bug upstream, or are 
you planning on upstreaming a diff?


Regards,

Jordan



On 2020-06-06 20:16, Philip Guenther wrote:
On Sat, Jun 6, 2020 at 5:08 PM Zé Loff <mailto:zel...@zeloff.org>> wrote:


On Sat, Jun 06, 2020 at 03:51:58PM -0700, Jordan Geoghegan wrote:
> I'm working on a simple awk snippet to convert the IP range data
listed in
> the Extended Delegation Statistics data from ARIN [1] and
convert it into
> CIDR blocks. I have a snippet that works perfectly fine on mawk
and gawk,
> but not on the base system awk. I'm 99% sure I'm not using any
GNUisms, as
> when I break the command up into two parts, it works perfectly.
>
> The snippet below does not work with base awk, but does work
with gawk and
> mawk: (Running on 6.6 -stable system)
>
>   awk -F '|' '{ if ( $3 == "ipv4" && $2 == "US")
printf("%s/%d\n", $4,
> 32-log($5)/log(2))}' delegated-arin-extended-latest.txt
>
>
> The command does output data, but it also throws errors for
certain lines:
>
>   awk: log result out of range
>   input record number 94027, file delegated-arin-extended-latest.txt
>   source line number 1
>
> Most CIDR blocks are calculated correctly, but about 10% of them
have errors
> (ie something that should calculated to be a /24 is instead
calculated to be
> a /30).

...

I have no idea about what is going on, but FWIW I can reproduce
this on
i386 6.7-stable and amd64 6.7-current (well, current-ish, #232).
Truncating the file to a single offending line produces the same
result:
log($5) is out of range.

It appears to have something to do with the last field. Removing it or
changing some of its characters seems to work, e.g.:


arin|US|ipv4|216.250.144.0|4096|20050503|allocated|5e58386636aa775c2106140445cf2c30

arin|US|ipv4|216.250.144.0|4096|20050503|allocated|5a58386636aa775c2106140445cf2c30
                                                    ^
Fails on the first line but works on the second.


Hah!  Nice observation!

The last field of the first line looks kinda like a number in 
scientific notation, but when awk internally tries to set up the 
fields it generates an ERANGE error...and the global errno variable is 
left with that value.  Several builtins in awk, including log(), 
perform operations and then check whether errno is set to EDOM or 
ERANGE but fail to clear errno beforehand.


The fix is to zero errno before all the code sequences that use the 
errcheck() function, ala:


--- run.c       13 Aug 2019 10:45:56 -      1.44
+++ run.c       7 Jun 2020 03:14:38 -
@@ -26,6 +26,7 @@ THIS SOFTWARE.
 #define DEBUG
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -1041,8 +1042,10 @@ Cell *arith(Node **a, int n)     /* a[0] + a
        case POWER:
                if (j >= 0 && modf(j, ) == 0.0)       /* pos integer 
exponent */

                        i = ipow(i, (int) j);
-               else
+               else {
+                       errno = 0;
                        i = errcheck(pow(i, j), "pow");
+               }
                break;
        default:        /* can't happen */
                FATAL("illegal arithmetic operator %d", n);
@@ -1135,8 +1138,10 @@ Cell *assign(Node **a, int n)    /* a[0] =
        case POWEQ:
                if (yf >= 0 && modf(yf, ) == 0.0)     /* pos integer 
exponent */

                        xf = ipow(xf, (int) yf);
-               else
+               else {
+                       errno = 0;
                        xf = errcheck(pow(xf, yf), "pow");
+               }
                break;
        default:
                FATAL("illegal assignment operator %d", n);
@@ -1499,12 +1504,15 @@ Cell *bltin(Node **a, int n)    /* builtin
                        u = strlen(getsval(x));
                break;
        case FLOG:
+               errno = 0;
                u = errcheck(log(getfval(x)), "log"); break;
        case FINT:
                modf(getfval(x), ); break;
        case FEXP:
+               errno = 0;
                u = errcheck(exp(getfval(x)), "exp"); break;
        case FSQRT:
+               errno = 0;
                u = errcheck(sqrt(getfval(x)), "sqrt"); break;
        case FSIN:
                u = sin(getfval(x)); break;


Todd, are we up to date with upstream, or is this latent there too?


Philip Guenther

Potential awk bug?

[Jordan Geoghegan]

Hello,

I was hoping the fine folks here could give me a quick sanity check, I'm 
by no means an awk guru, so I'm likely missing something obvious. I 
wanted to ask here quickly before I started flapping my gums on bugs@.


I'm working on a simple awk snippet to convert the IP range data listed 
in the Extended Delegation Statistics data from ARIN [1] and convert it 
into CIDR blocks. I have a snippet that works perfectly fine on mawk and 
gawk, but not on the base system awk. I'm 99% sure I'm not using any 
GNUisms, as when I break the command up into two parts, it works perfectly.


The snippet below does not work with base awk, but does work with gawk 
and mawk: (Running on 6.6 -stable system)


  awk -F '|' '{ if ( $3 == "ipv4" && $2 == "US") printf("%s/%d\n", $4, 
32-log($5)/log(2))}' delegated-arin-extended-latest.txt



The command does output data, but it also throws errors for certain lines:

  awk: log result out of range
  input record number 94027, file delegated-arin-extended-latest.txt
  source line number 1

Most CIDR blocks are calculated correctly, but about 10% of them have 
errors (ie something that should calculated to be a /24 is instead 
calculated to be a /30).


However, when I break it up into two parts, it produces the expected output:

  awk -F '|' '{ if ( $3 == "ipv4" && $2 == "US") print($4, $5)}' 
delegated-arin-extended-latest.txt | awk  '{printf("%s/%d\n", $1, 
32-log($2)/log(2)) }'


As you can see, the same number of lines are printed, but the hashes are 
different.


  luna$ gawk -F '|' '{ if ( $3 == "ipv4" && $2 == "US") 
printf("%s/%d\n", $4, 32-log($5)/log(2))}' delegated-*-latest.txt | wc -l

     56446
  luna$ mawk -F '|' '{ if ( $3 == "ipv4" && $2 == "US") 
printf("%s/%d\n", $4, 32-log($5)/log(2))}' delegated-*-latest.txt | wc -l

     56446
  luna$ awk -F '|' '{ if ( $3 == "ipv4" && $2 == "US") 
printf("%s/%d\n", $4, 32-log($5)/log(2))}' delegated-*-latest.txt 
2>/dev/null | wc -l

     56446

  luna$ awk -F '|' '{ if ( $3 == "ipv4" && $2 == "US") 
printf("%s/%d\n", $4, 32-log($5)/log(2))}' 
delegated-arin-extended-latest.txt 2>/dev/null | md5

    6f549bbc0799bc202c12695f8530d1df
  luna$ gawk -F '|' '{ if ( $3 == "ipv4" && $2 == "US") 
printf("%s/%d\n", $4, 32-log($5)/log(2))}' 
delegated-arin-extended-latest.txt 2>/dev/null | md5

    40c28b8ebfd2796e1ae15d9f6401c0c1
  luna$ mawk -F '|' '{ if ( $3 == "ipv4" && $2 == "US") 
printf("%s/%d\n", $4, 32-log($5)/log(2))}' 
delegated-arin-extended-latest.txt 2>/dev/null | md5

    40c28b8ebfd2796e1ae15d9f6401c0c1


Example of the differences:

--- mawk.txt    Sat Jun  6 18:43:30 2020
+++ awk.txt Sat Jun  6 18:43:38 2020
@@ -29,7 +29,7 @@
 9.64.0.0/10
 9.128.0.0/9
 11.0.0.0/8
-12.0.0.0/8
+12.0.0.0/30
 13.0.0.0/11
 13.32.0.0/12
 13.48.0.0/14
@@ -415,7 +415,7 @@
 23.90.64.0/20
 23.90.80.0/21
 23.90.88.0/22
-23.90.92.0/22
+23.90.92.0/30
 23.90.96.0/19
 23.91.0.0/19
 23.91.32.0/19
@@ -545,8 +545,8 @@
 23.133.224.0/24
 23.133.240.0/24
 23.134.0.0/24
-23.134.16.0/24
-23.134.17.0/24
+23.134.16.0/30
+23.134.17.0/30


Any insight or advice would be much appreciated.

Regards,

Jordan

[1] https://ftp.arin.net/pub/stats/arin/delegated-arin-extended-latest

Re: Filling a 4TB Disk with Random Data

[Jordan Geoghegan]

On 2020-06-01 06:58, Justin Noor wrote:

Hi Misc,

Has anyone ever filled a 4TB disk with random data and/or zeros with
OpenBSD?

How long did it take? What did you use (dd, openssl)? Can you share the
command that you used?

Thank you so much



I've used OpenBSD to overwrite up to 8TB disks. I use a large block size 
with 'dd' and make sure to use /dev/rsdX (the 'r' makes things much 
faster).

Re: 10Gbps X520 network adapter only passing 3.5Gbps

[Jordan Geoghegan]

On 2020-05-06 04:04, Stuart Henderson wrote:

On 2020-05-06, Jordan Geoghegan  wrote:


On 2020-05-04 06:42, Kalle Kadakas wrote:

Greetings OpenBSD community,

I am running into severe bandwidth limitations whilst passing traffic
   through an OpenBSD firewall.
The NIC in use is an Intel 10Gb 2-port X520 adapter from which I would
   hope to pass through at least 7Gbps+, yet the best results I have
   gotten is only around 3.5Gbps.

The results of bandwidth measurements (iperf for 30sec...
   

As has been discussed on misc previously, iperf is not suitable for
benchmarking networking throughput on OpenBSD. It ends up just
benchmarking the gettimeofday syscall (something that is cheap on Linux,
but relatively expensive on OpenBSD I'm told).

clock_gettime. It's iperf3 that calls this often; iperf not so much.
But when testing with default options, you may see higher numbers from
iperf3: the direct comparison isn't fair though because it uses 128K
TCP buffers by default, whereas iperf uses the OS default.

On Linux clock_gettime often doesn't use a system call, on OpenBSD it
does (and with some of the mitigations for cpu bugs, system calls are
more expensive than they used to be).


For best results, use tcpbench for your OpenBSD networking benchmarks.

For accurate results of packet forwarding performance, use fast packet
sources/sinks running whatever OS either side of an OpenBSD router.




Thanks for clarifying Stuart, I knew I was about 70% of the way there.

Jordan

Re: 10Gbps X520 network adapter only passing 3.5Gbps

[Jordan Geoghegan]

On 2020-05-04 06:42, Kalle Kadakas wrote:

Greetings OpenBSD community,

I am running into severe bandwidth limitations whilst passing traffic
  through an OpenBSD firewall.
The NIC in use is an Intel 10Gb 2-port X520 adapter from which I would
  hope to pass through at least 7Gbps+, yet the best results I have
  gotten is only around 3.5Gbps.

The results of bandwidth measurements (iperf for 30sec...
  


As has been discussed on misc previously, iperf is not suitable for 
benchmarking networking throughput on OpenBSD. It ends up just 
benchmarking the gettimeofday syscall (something that is cheap on Linux, 
but relatively expensive on OpenBSD I'm told). For best results, use 
tcpbench for your OpenBSD networking benchmarks.

Re: SpeedTest-cli results accuracy ?

[Jordan Geoghegan]

Hi Kanto,

The Edgerouter Lite will not push much more than 200mbps, so that will 
certainly be a bottleneck. The only reason the ERlite can push 1Gbit 
with stock firmware is because of proprietary cut through routing and 
other garbage -- what they dont tell you is that as soon as you enable 
QoS or any fancy firewalling (or even IPv6!) you lose that hardware 
acceleration and all forwarding/packet handling is done purely on the CPU.


With regards to speedtest-cli, those results are not accurate. The only 
way to be sure it to use something like tcpbench and test it against 
multiple locations (I usually spin up a few vultr VPS in various 
locations to confirm my speeds for example). Also, I'm sure you already 
know this, but you also should never run the benchmarking program on 
your router, as that will obviously skew the results.


Cheers,

Jordan



On 2020-05-05 17:47, Kanto Andria wrote:

Hello all,
First post here. So please be indulgent ;-)). My question is about the 
speedtest-cli tool and the tests results with OpenBsd.Let me explain. I have 
multiple machines - physical and virtual - mix of BSD and Linux - and I am in a 
process of rebuilding my Firewall - obviously with OpenBSD/PF.
I have had an old Firewall using EdgeRouterLite which is broken now (no 
response from keyboard input using the console access - different story).With 
the ERL FW, when I increase  the ISP contract speeds from  200/30 Mbps to 
400/50 Mbps - doing a speed using any computer from the LAN did not pass over 
around ~220/35 Mbps.
The provider provided a Zyxel (if this matters) which "acts temporarily" as the 
Firewall + DHCP, etc. Any speedtest from Linux, Windows (son's game computer) got around 
the 400 Mbps/50Mbps or more.
The OpenBSD station (running 6.6) gets no more than 250 Mbps - the new Firewall 
I'm building shows the same results (see dmesg below) - no GUI for the 2 
machines - using speedtest-cli.
Following are the tests - machines - I ran with their respective results:
- Lenovo ThinkCenter - OpenBSD 6.6 - speedtest-cli : ~230/37 Mbps- Future 
Firewall (acting as workstation for the test)  - OpenBSD 6.6: around the same 
results- OpenBSD 6.6 running as VM on Manjaro Linux - around the same results
- Manjaro Linux (Physical) hosting the OBSD VM - reach around or more the 
~420/52 Mbps (using speedtest-cli and the Browser)- PFSense running as VM on 
Manjaro Linux - where I installed speedtest-cli - reach around the ~400/50 Mbps
Even they are not the same tools (iperf vs speedtest-cli) - running iperf3 
between OBSD vs OBSD/Linux and/or tcpbench, the tests display results close to 
960 Mbps.

So my question is can: I rely on the on the speedtest results? What else should 
I verify? Changing cables/direct connections to the current router were already 
done.

Thanks for any inputs and clarification.
Kanto

dmesg for the future Router/Firewall
OpenBSD 6.6 (GENERIC.MP) #8: Fri Apr 17 15:06:32 MDT 2020
     
r...@syspatch-66-amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 8489873408 (8096MB)
avail mem = 8219873280 (7839MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.8 @ 0xec410 (83 entries)
bios0: vendor American Megatrends Inc. version "5.6.5" date 06/30/2018
bios0: INTEL Corporation Q3XXG4-P
acpi0 at bios0: ACPI 5.0
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP APIC FPDT FIDT MCFG HPET SSDT UEFI SSDT ASF! SSDT SSDT 
SSDT DMAR
acpi0: wakeup devices PEG0(S4) PEGP(S4) PEG1(S4) PEGP(S4) PEG2(S4) PEGP(S4) 
RP01(S4) PXSX(S4) RP02(S4) PXSX(S4) RP03(S4) PXSX(S4) RP04(S4) PXSX(S4) 
RP05(S4) PXSX(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM) i3-4030U CPU @ 1.90GHz, 1895.92 MHz, 06-45-01
cpu0: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4.1.1.1, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Core(TM) i3-4030U CPU @ 1.90GHz, 1895.62 MHz, 06-45-01
cpu1: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu1: 

Re: More than 16 partitions

[Jordan Geoghegan]

On 2020-04-23 11:45, zeurk...@volny.cz wrote:

"Jan Betlach"  wrote:

For a non-native English speaker like myself, it is very difficult to
read your mestuff...

Your point is well-taken (though this is just the way mespeaks); yet,
Theo is a native speaker, and he seems to have completely missed the
content of merecent responses.

Weird, isn't it? Anyway, as this would appear to be quite OT, me'd
suggest we continue this (if at all) in private mail...


Jan

Take care,

 --zeurkous.



Say Hi to Boss Nass for me next time you're in Otoh Gunga.

Jordan

Re: Openbsd mirrors in Kazakhstan?

[Jordan Geoghegan]

On 2020-04-10 10:11, Nikita Stepanov wrote:

Openbsd mirrors in Kazakhstan?


The Russian mirror is hosted by Yandex. Moscow is likely the closest 
mirror to Kazakhstan, otherwise I would recommend trying one of the CDN 
options

Re: Hosting a CDN question

[Jordan Geoghegan]

On 2020-03-16 03:26, Flipchan wrote:

Hey all,

My company needs to put up a cdn for fast hosting of javascript, images and css 
for websites, and then i would need something faster then httpd.


Does anyone here run a cdn for static website content?

If so what software did u use to set it up ?

have a good one
Sincerely
Filip


What level of traffic are you looking to push? Have you done any tests 
to confirm httpd was unsuitable or was the bottleneck?


When coupled with relayd, you can have a very powerful setup. I have a 
setup with relayd + httpd on modest hardware, and I can push over 5,000 
requests per second.

Re: Jitsi on OpenBSD

[Jordan Geoghegan]

On 2020-03-16 03:59, Edd Barrett wrote:

Hi,

(CC people who may be knowledgable in this area)

I was wondering if anyone has got the Jitsi (https://jitsi.org/)
web-client working on OpenBSD?

It's open-source (and self-hostable) video conferencing.

No prizes for guessing why I'm investigating this :P

I've just (quickly) tried the browser client in firefox:

  - It recognises my microphone and my camera.
  - Thumbnail shows local video feed OK.
  - I can hear audio from an android participant.
  - The android participant cannot hear the audio from the OpenBSD machine.
  - The video is super-flaky on both ends.

Did this, as per firefox README:

  - I have sysctl kern.audio.record=1.
  - I chowned /dev/video0 to me.

This evening I'm going to have a deeper play around (e.g. verify if mic
works in aucat), but if anyone has got this working before, I'd love to
hear what tweaks they had to do.

Could be that the jitsi server is overloaded.

Thanks

You may also want to try out talky.io, its treated my me well over the 
years and has a very similar concept to jitsi.


Jordan

(ps sorry for sending you this twice, forgot to cc misc on my first attempt)

Re: riscv

[Jordan Geoghegan]

On 2020-03-14 23:19, Mike Larkin wrote:

On Sat, Mar 14, 2020 at 11:18:11PM -0700, Mike Larkin wrote:

On Fri, Mar 13, 2020 at 02:12:19PM -0700, Jordan Geoghegan wrote:


On 2020-03-13 09:50, Christian Weisgerber wrote:

On 2020-03-13, "Peter J. Philipp"  wrote:


Any developer working on a riscv port and willing to share their unofficial
work for possible future collaboration?

I think I'd have heard by now if somebody was, so I'll go out on a
limb and say no, nobody's working on a RISC-V port.


I stumbled across this a while back, this guy at least claims to be
attempting a port to RISC-V...

https://github.com/MengshiLi/openbsd-riscv-notes


We have a riscv64 kernel booting up to the rootdev prompt, and are working on

PS, "We" here is my student team. This is not being done as part of the main
OpenBSD development effort. We hope to be able to get this committed when it is
ready but we are nowhere near that yet.


getting plic working so that we can use virtio disks.

The link above is from one of my students that is working on this. This is not
in the main tree, and I'm not sure what it will take to get it there (we are
using a newer version of clang than is in base).

-ml



That's great to hear, I'm excited to see what comes of this work, hats 
off to you and your students!


It's a shame that clang was re-licensed, I hope we don't end up with a 
repeat of the gcc 4.x saga. I've heard a number of people lamenting 
about various improvements made in clang post relicencing, especially 
for non x86 arches such as powerpc.


Jordan

Re: experience setting up a low memory machine

[Jordan Geoghegan]

On 2020-03-13 18:31, Stuart Henderson wrote:

On 2020-03-13, Jordan Geoghegan  wrote:

I wouldn't get too excited about running on low memory machines. The
more RAM you can throw at something, the better, as this allows more
cache room as well as improving function of ASLR and other memory
randomizations.

It does allow more cache, but what matters for ASLR is address space, not RAM.



Ah okay, you're right. I was remembering reading something about low 
memory conditions weakening ASLR, but after doing a quick check, it 
seems it was an implementation specific issue with Windows and Linux.

Re: experience setting up a low memory machine

[Jordan Geoghegan]

On 2020-03-11 19:20, Aaron Mason wrote:

On Wed, Mar 11, 2020 at 6:47 PM Jordan Geoghegan  wrote:



On 2020-03-11 00:13, Stuart Longland wrote:

On 15/2/20 6:43 pm, Dumitru Moldovan wrote:

[SNIP]

[SNIP]

Sometimes it's better to realise when something has past its prime.

A year or two ago I had OpenBSD working on my iBook with 64MB of RAM,
even got FVWM working on it. For fun and testing purposes, I ran some
small OpenBSD virtual machines with 64MB RAM as well. A few years back I
got OpenBSD to boot with 32MB, but it wasn't particularly usable. I've
found 128MB to be usable for basic terminal work, but you're definitely
correct about 256MB being the bare minimum for anything fancy or GUI
related.



At work I run OpenBSD 6.1 in a VM for Request Tracker.  It has 512MB
RAM and it seems that may very well be overkill.  At previous jobs I
can ManageEngine ServiceDesk Plus and even in Linux you needed 2GB
minimum just for it to get out of bed.  I plan on rebuilding it with
6.6 (can't update RT because packages are too old in 6.1) and might
run it on 256MB for shits and giggles.



I wouldn't get too excited about running on low memory machines. The 
more RAM you can throw at something, the better, as this allows more 
cache room as well as improving function of ASLR and other memory 
randomizations.

Re: riscv

[Jordan Geoghegan]

On 2020-03-13 09:50, Christian Weisgerber wrote:

On 2020-03-13, "Peter J. Philipp"  wrote:


Any developer working on a riscv port and willing to share their unofficial
work for possible future collaboration?

I think I'd have heard by now if somebody was, so I'll go out on a
limb and say no, nobody's working on a RISC-V port.



I stumbled across this a while back, this guy at least claims to be 
attempting a port to RISC-V...


https://github.com/MengshiLi/openbsd-riscv-notes

Re: pf-badhost-0.3 released

[Jordan Geoghegan]

On 2020-03-11 12:41, Anders Andersson wrote:

On Tue, Mar 10, 2020 at 10:53 PM Jordan Geoghegan  wrote:

pf-badhost and unbound-adblock are both now at version 0.3, released
earlier today.

Links to the scripts can be found here:

www.geoghegan.ca/pfbadhost.html
www.geoghegan.ca/unbound-adblock.html

Thanks, this looks very interesting! But maybe you can help answering
a question that popped up when I read your page about pf-badhost.

You mention that "Subnet aggregation is used to take the address list
and "aggregate" the addresses into the smallest possible
representation using CIDR blocks.", but I was under the assumption
that pf already did this for its tables to speed up lookups.

Is there anything preventing the aggregation code to run on every pf
table modification? Assuming an already sorted list, it shouldn't take
long to merge a new entry. Perhaps I've missed some use of pf tables
that makes this impossible or not applicable in the general case.



Hi Anders,

I am by no means an expert on the nuts and bolts of pf, but I do know 
that pf stores table data in a radix tree / radix table. By their 
nature, radix trees ignore exact duplicates, but I'm not exactly sure 
how they handle the partial overlapping of ranges. This article gives an 
easy to follow cursory overview of raddix trees if you're interested:

https://blog.sqreen.com/demystifying-radix-trees/

As far as I understand, pf makes no modifications to the contents of 
your tables, all it does is parse the list to confirm the addresses 
and/or CIDR blocks are valid. When it's looking for matches within 
ranges, it will look for the most specific match available. For example, 
if you have a list containing an overlap:

...
192.168.0.0/16
192.168.1.0/22
...
When a packet from 192.168.1.5 arrives and is processed by a rule 
referencing this table, it will match with 192.168.1.0/22. Even though 
both entries are valid and match the packet, the /22 is more specific, 
and thus the one which matches closest.


pf may do some magic optimizations under the hood that I'm unaware of, 
but at the end of the day, it does not modify the actual contents of 
your table.


The use I've found in the subnet aggregation function has been mostly 
for the purpose of keeping the list clean and tidy. I have a few 
installations where I have all the lists enabled, including the use of 
the GeoIP country blacklisting function. On these installations, subnet 
aggregation can reduce the /etc/pf-badhost.txt file from ~60,000 lines 
down to ~40,000 lines. For example, when blocking China's netblocks 
(which pulls an aggregated list of all addresses assigned to China by 
APNIC, and thus uses massive CIDR blocks of /10's etc), if any addresses 
from any of the other blocklists come from China, they will be removed 
from the list as they are already covered by the CIDR block info from 
APNIC. I run pf-badhost on a bunch of Edgerouter Lites, and I've found 
them to run better when the lists are tidy.


With regards to pf performing aggregation on all tables automatically, 
it wouldn't make sense to run the full subnet aggregation calculations 
for every table load or insertion/removal, as it can be quite CPU 
intensive. It takes less than a second to load the table on a $5 Vultr 
VPS, it takes 20-70 seconds to run the subnet aggregation (depending on 
which lists are enabled). On my Edgerouter Pro with all the lists 
enabled, it takes ~6 minutes. On my Edgerouter Lite it takes ~15 minutes 
to run (over 2 hours when using the built in Perl-based aggregator). I 
just run the aggregation function with nice and let it do its thing, its 
being called by cron in the wee hours, so I'm fine just letting it chug 
along.


Regards,

Jordan

Re: experience setting up a low memory machine

[Jordan Geoghegan]

On 2020-03-11 00:13, Stuart Longland wrote:

On 15/2/20 6:43 pm, Dumitru Moldovan wrote:

Not really, about 21 years ago I was learning to get XFree86 working,
to break free from the console on a desktop with 24MB of RAM.

It's all relative… I can recall years ago experimenting with operating
systems on old machines (even by that day's standards)… trying to get
Slackware Linux running in 4MB RAM.

Today, I'm trying to cram stuff into 32kB of RAM and thinking how 4MB
and a MMU would be luxury!  (Even RetroBSD needs 128kB.)

I'm sure someone on here will tell me how they used to flip switches on
a front panel to fill the whopping 128 bytes of RAM on their 8080 with
machine code whilst dreaming of a punched tape loader and a few more
DRAM chips.

I would suggest any "modern" multi-tasking general purpose OS such as
OpenBSD, Linux, etc, you're looking at a minimum of 256MB RAM.  In the
future, probably consider doubling that… and again.

I have an old PII 300MHz laptop with 160MB RAM, and while it boots Linux
just fine (I had it running Gentoo doing AX.25 packet radio stuff),
firing up a web browser (Firefox) to check the weather is unusably slow.

I could throw OpenBSD on it, but not sure that would really make much
difference: kernel re-linking is going to be painful on that machine.

Sometimes it's better to realise when something has past its prime.


A year or two ago I had OpenBSD working on my iBook with 64MB of RAM, 
even got FVWM working on it. For fun and testing purposes, I ran some 
small OpenBSD virtual machines with 64MB RAM as well. A few years back I 
got OpenBSD to boot with 32MB, but it wasn't particularly usable. I've 
found 128MB to be usable for basic terminal work, but you're definitely 
correct about 256MB being the bare minimum for anything fancy or GUI 
related.

pf-badhost-0.3 released

[Jordan Geoghegan]

Hey folks,

Last time I posted about this, I got a fair bit of interest and I've had 
quite a few downloads and enquiries about pf-badhost, so I figured I'd 
share here that I've updated the script.


pf-badhost and unbound-adblock are both now at version 0.3, released 
earlier today.


I highly encourage anybody running an older version of these scripts to 
update to the latest version, as I have made a number of significant 
improvements to the security and robustness of the script.


Links to the scripts can be found here:

www.geoghegan.ca/pfbadhost.html
www.geoghegan.ca/unbound-adblock.html

Regards,
Jordan

Re: Time jumping forward issue under OpenBSD 6.6 VMM

[Jordan Geoghegan]

On 2020-03-03 07:43, mabi wrote:

Hello,

I am running an OpenBSD 6.6 VMM server with a few virtual machines also running 
6.6 and noticed that one of them which is running Dovecot 2.3.9 package outputs 
the following warning every few 10-15 seconds:

dovecot: imap: Warning: Time jumped forwards 28.294585 seconds

It looks like there is a time issue on that VM although I am running the 
default ntpd of OpenBSD 6.6 and I have added the following parameter into my 
/etc/sysctl.conf on that VM:

kern.timecounter.hardware=tsc

Is there anything else I can do to avoid this time issue in my VM?

Best regards,
Mabi



The clocks are basically broken on vmm. The pvclock stuff is definitely 
an improvement, but it's still not there. There's still a ways to go 
before we have proper, reasonably accurate clocks for vmm guests.

Re: suggestions for USB printer (maybe even with scanner)?

[Jordan Geoghegan]

On 2020-02-18 21:28, Stuart Longland wrote:

On 6/2/20 7:26 am, Adam Thompson wrote:

I don't know what you need in a printer, and I don't know what you mean
by cheap, so... YMMV.

However, I've found Brother **LASER** printers to be very good, and most
of them support PCL6 and/or PS3.
For example, the HL-L2370DW can only connect via USB, and supports PCL6,
and currently sells for ~C$150-160.

Somewhat related, does this hold true for their Ethernet-connected
printers as well?

We have an Epson WF-7510 which is due for replacement and while I can
set up OpenBSD on a small box to act as a print server, having something
I can point OpenBSD's lpd or cups at over a network is helpful too.


Yes, I have a Brother laser printer that I print to using lpd. I can 
send it PDF or postscript documents (or raw text) via lpr and it prints 
just fine, without any drivers or cups or other such nonsense.

Re: rspamd stop rc script doesn't work in OpenBSD 6.6

[Jordan Geoghegan]

On 2020-02-09 11:46, aisha wrote:
You need to use pkill -9 to kill rspamd, which i think should be added 
to the stop part of the rspamd daemon.


At least this is what I have been using, any other methods would be 
nice to know.




You dont need to restart rspamd if you're just modifying a config file.

You can just run "rcctl reload rspamd" to reload your config files if 
you've made any changes.


The rspamd maintainer has acknowledged the issue, and the restarting 
issue is fixed in current.

Re: Low throughput with 1 GigE interface

[Jordan Geoghegan]

On 2020-01-30 14:29, Christian Weisgerber wrote:

On 2020-01-30, Jordan Geoghegan  wrote:


All you're doing is benchmarking the speed of iperf on that machine.

I vaguely remember a thread somewhere that concluded that one of
these network benchmark tools degenerated into a benchmark of
gettimeofday(2), which apparently is very cheap on Linux and not
cheap on OpenBSD.  So you end up measuring the performance of this
system call.

I don't remember whether it was iperf...



That sounds about right. I vaguely remember reading a thread about iperf 
on misc some time in the past year mentioning that.


While OpenBSD obviously doesn't have the same network performance as 
Linux or FreeBSD, as work continues on unlocking more of the kernel, 
things will continue to get better. I think bluhm@ regularly runs some 
automated benchmarks that show that OpenBSD maxes out at around 4-5 Gbit 
/ second throughput.

Re: Low throughput with 1 GigE interface

[Jordan Geoghegan]

On 2020-01-30 13:14, Jordan Geoghegan wrote:

On 2020-01-30 10:06, livio wrote:

@KatolaZ and @remi

Thank you for your inputs on iperf2 vs. iperf3.

After all the tests I needed a clean setup again and reinstalled both
OpenBSD and Window 10.

With the new notebook (Dell vs Lenovo) I have different results.
Dell: ~ 200Mbit/s
Lenovo: ~ 145Mbit/s

iperf2 vs. iperf3 (I also ran the corresponding version on Windows):


[snip]

All you're doing is benchmarking the speed of iperf on that machine. 
Try running the tests with tcpdump.


If this APU is meant to be a router, then you should be testing its 
forwarding performance.



Woops, I meant to say tcpbench, not tcpdump.

Re: Low throughput with 1 GigE interface

[Jordan Geoghegan]

On 2020-01-30 10:06, livio wrote:

@KatolaZ and @remi

Thank you for your inputs on iperf2 vs. iperf3.

After all the tests I needed a clean setup again and reinstalled both
OpenBSD and Window 10.

With the new notebook (Dell vs Lenovo) I have different results.
Dell: ~ 200Mbit/s
Lenovo: ~ 145Mbit/s

iperf2 vs. iperf3 (I also ran the corresponding version on Windows):


[snip]

All you're doing is benchmarking the speed of iperf on that machine. Try 
running the tests with tcpdump.


If this APU is meant to be a router, then you should be testing its 
forwarding performance.

Re: dig -p 5353 foo.bar core dumped

[Jordan Geoghegan]

On 2020-01-21 11:32, Paul de Weerd wrote:

On Tue, Jan 21, 2020 at 11:25:33AM -0800, Jordan Geoghegan wrote:
| dig doesnt core dump for me, it just prints this warning: ";; Error,
| only port 53 supported". I wonder why the error isnt being printed
| for OP.

Interesting.  Are you on -current?  I updated to the latest snapshot
yesterday and get this:

[weerd@pom] $ dig -p 1234 foo.bar @127.0.0.1
Abort trap
[weerd@pom] $ dmesg | tail -n1
dig[69548]: pledge "dns", syscall 28

You may be doing something similar yet different?

Cheers,

Paul 'WEiRD' de Weerd



I ran that command on a machine running 6.6 stable, so thats probably why.

probook$ dig -p 1234 foo.bar @127.0.0.1
;; Error, only port 53 supported

Re: dig -p 5353 foo.bar core dumped

[Jordan Geoghegan]

On 2020-01-21 11:05, Paul de Weerd wrote:

On Tue, Jan 21, 2020 at 06:58:02PM +0100, Dieter Rauschenberger wrote:
| Hi misc,
|
| on my intranet i have unbound and nsd running, both on the same
| machine. unbount is listening on port 53, nsd is listening on port
| 5353 on 127.0.0.1. If i run
|
| dig @127.0.0.1 -p 53 foo.bar
|
| everything is fine. But if I want to query nsd direct with
|
| dig @127.0.0.1 -p 5353 foo.bar
|
| dig core dumps and I can read the following line in /var/log/messages:

That's pledge, as the below kernel message explains.

| Jan 21 18:42:44 ws /bsd: dig[59239]: pledge "dns", syscall 28

Dig is only allowed to do port 53.  This can be worked around by using
dig from the bind port, or with a patch to dig that allows you to use
other ports when -p is given on the command line.

However, I'd recommend running nsd on port 53 on 127.0.0.1; there it
doesn't interfere with unbound (assuming that's not listening on
127.0.0.1).  Alternatively, you can easily add a second IP address on
your loopback interface (::2) and have nsd listen there.

| The issue can be easily reporduced with any other port. No need to
| have a nsd running.

That's because it has nothing to do with nsd but with dig.  Dig has
promised not to use ports other than 53, and then when you do use
ports other than 53, the kernel terminates the process.

Cheers,

Paul 'WEiRD' de Weerd



dig doesnt core dump for me, it just prints this warning: ";; Error, 
only port 53 supported". I wonder why the error isnt being printed for OP.

Re: Can't locate OpenBSD/Quirks.pm in @INC

[Jordan Geoghegan]

On 2020-01-17 18:10, myml...@gmx.com wrote:

HI,


I downloaded the install66.fs snapshot today, 20200117, and did a fresh
install.  Even though I got the full install set, i used http from
ftp.openbsd.org as the install source.

Installation went fine but when I tried to install packages I get the
above error.

"# pkg_add -vn pftop
quirks-3.216 signed on 2020-01-17T19:15:00Z
quirks-3.216: ok
Can't load quirk: Can't locate OpenBSD/Quirks.pm in @INC (you may need
to install the OpenBSD::Quirks module) (@INC contains:
/usr/local/libdata/perl5/site_perl/amd64-openbsd
/usr/local/libdata/perl5/site_perl /usr/libdata/perl5/amd64-openbsd
/usr/libdata/perl5) at /usr/libdata/perl5/OpenBSD/AddDelete.pm line 350.

pftop-0.7p19: ok
Merging manpages in /usr/local/man: /usr/local/man/man8/pftop.8
Extracted 252817 from 253475"


[snip]

I believe quirks gets automatically installed when you install your 
first package.

Re: OpenBSD's extremely poor network/disk performance?

[Jordan Geoghegan]

On 2020-01-09 06:22, Hamd wrote:

Joe, are you a joke? Please stop insulting me, this is not
my/your_personal_fancy_forum.

This will be my last post here in misc.

Default setups, no config. changes.
Just patches installed.
Same hardware.

FreeBSD:
freebsd@test:~ # time sh -c "dd if=/dev/zero of=test.tmp bs=4k count=5
&& sync"
5+0 records in
5+0 records out
20480 bytes transferred in 0.239590 secs (854792500 bytes/sec)
0.000u 0.195s 0:00.25 76.0% 22+198k 0+1568io 0pf+0w

Result: *854.79 MB/s disk speed*

freebsd@test:~ # uname -a
FreeBSD test.local 12.1-RELEASE-p1 FreeBSD 12.1-RELEASE-p1 GENERIC  amd64

OpenBSD:
test$ time sh -c "dd if=/dev/zero of=test.tmp bs=4k count=5 && sync"
5+0 records in
5+0 records out
20480 bytes transferred in 12.303 secs (16645247 bytes/sec)
 0m12.32s real 0m00.13s user 0m01.28s system

Result: *16.64 MB/s disk speed*



[snip]

probook$ dd if=/dev/zero of=test.tmp bs=4k count=5 && sync
5+0 records in
5+0 records out
20480 bytes transferred in 0.731 secs (280047607 bytes/sec)

I'm getting well over 250MB/s on my laptop.

Is your hard drive attached as an sd or wd device?

Re: Odd /tmp behavior

[Jordan Geoghegan]

On 2020-01-07 11:06, Karel Gardas wrote:



On 1/7/20 7:38 PM, Jordan Geoghegan wrote:

 > Using softdep on /tmp is a silly idea. >
Why? To naive eyes it may look like a natural solution: e.g. before 
temp file is even created (on drive), it may be deleted which means 
there is no meta-data change hence speedup of operation on /tmp. In 
case of classical ffs, you will need to create file (sync meta-data 
update), save some data (async), delete file (sync meta-data update). 
But honestly still need to read the code...


softdep is very complex, and it is by no means perfect or bug free. A 
good rule of thumb is to only use softdep on larger partitions that will 
see a large number of writes (particularly smaller and/or random writes).


Softdep can chew up a fair amount of kernel memory as well, and in 
certain cases changes can take over a minute to actually make their way 
on to disk. If softdep was the magic bullet that some people think it 
is, it would be enabled by default.

Re: LibreSSL performance issue

[Jordan Geoghegan]

Is there a specific reason you're running i386 instead of amd64? And why 
are you testing this on FreeBSD? Wrong mailing list


On 2020-01-07 08:26, Joe Greco wrote:

On Tue, Jan 07, 2020 at 09:33:46AM -0600, Edgar Pettijohn wrote:

In reality, when you dig down, often you find that there's another
reason for the issue.?? I was recently trying to substitute libressl
into an openssl environment.?? Performance tanked.?? Some checking
showed the speed of "speed -evp aes-256-gcm" was way off.?? It looked
to me like it was an issue with not using AES-NI.?? I'm not going to
blame libressl for that, I just lacked the time to do a deep dive on
it to figure out what was (hopefully!) configured wrong.?? Probably
something with ia32cap or whatever the libressl equivalent is.

... JG

I believe it has something to do with actually zeroing out memory
before freeing it. Which seems like a good thing to do for crypto
stuff.

My apologies.  I posted an insufficient description of the issue as it
was intended as an argument refuting the OP.  If we want to discuss my
issue, that's fine and I welcome the input.  I normally manage to
resolve these things eventually but this stumped me a bit.

This appears to be an i386-specific issue and it is perhaps a 5:1
performance difference.

Compiled on a FreeBSD 12.1R-amd64 VM, I see exactly what I would hope
to see:

--Begin-FreeBSD-12.1R-amd64
# uname -r; uname -m
12.1-RELEASE
amd64

# libressl-3.0.2/apps/openssl/openssl speed -evp aes-256-gcm
WARNING: can't open config file: /usr/local/etc/ssl/openssl.cnf
Doing aes-256-gcm for 3s on 16 size blocks: 42776805 aes-256-gcm's in 3.00s
Doing aes-256-gcm for 3s on 64 size blocks: 28274190 aes-256-gcm's in 3.01s
Doing aes-256-gcm for 3s on 256 size blocks: 9382555 aes-256-gcm's in 3.01s
Doing aes-256-gcm for 3s on 1024 size blocks: 2636912 aes-256-gcm's in 3.01s
Doing aes-256-gcm for 3s on 8192 size blocks: 334132 aes-256-gcm's in 3.01s
LibreSSL 3.0.2
built on: date not available
options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) 
blowfish(idx)
compiler: information not available
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes256 bytes   1024 bytes   8192 bytes
aes-256-gcm 228204.73k   601456.74k   798353.68k   897432.60k   909765.10k

# openssl speed -evp aes-256-gcm
Doing aes-256-gcm for 3s on 16 size blocks: 40297566 aes-256-gcm's in 3.01s
Doing aes-256-gcm for 3s on 64 size blocks: 27287454 aes-256-gcm's in 3.01s
Doing aes-256-gcm for 3s on 256 size blocks: 10106391 aes-256-gcm's in 3.01s
Doing aes-256-gcm for 3s on 1024 size blocks: 2858781 aes-256-gcm's in 3.01s
Doing aes-256-gcm for 3s on 8192 size blocks: 368695 aes-256-gcm's in 3.00s
Doing aes-256-gcm for 3s on 16384 size blocks: 184909 aes-256-gcm's in 3.01s
OpenSSL 1.1.1d-freebsd  10 Sep 2019
built on: reproducible build, date unspecified
options:bn(64,64) rc4(16x,int) des(int) aes(partial) idea(int) blowfish(ptr)
compiler: clang
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes256 bytes   1024 bytes   8192 bytes  
16384 bytes
aes-256-gcm 214362.12k   580620.32k   860172.00k   973262.71k  1006783.15k  
1007226.70k
--End-FreeBSD-12.1R-amd64

Okay, so that looks fantastic to me.  Now running it on i386 on
a VM "right next door" on the same hypervisor.

--Begin-FreeBSD-12.1R-i386
# uname -r; uname -m
12.1-RELEASE
i386

# libressl-3.0.2/apps/openssl/openssl speed -evp aes-256-gcm
WARNING: can't open config file: /usr/local/etc/ssl/openssl.cnf
Doing aes-256-gcm for 3s on 16 size blocks: 8904897 aes-256-gcm's in 3.00s
Doing aes-256-gcm for 3s on 64 size blocks: 2387064 aes-256-gcm's in 3.01s
Doing aes-256-gcm for 3s on 256 size blocks: 603284 aes-256-gcm's in 3.01s
Doing aes-256-gcm for 3s on 1024 size blocks: 153381 aes-256-gcm's in 3.01s
Doing aes-256-gcm for 3s on 8192 size blocks: 19041 aes-256-gcm's in 3.01s
LibreSSL 3.0.2
built on: date not available
options:bn(64,32) rc4(ptr,int) des(idx,cisc,16,long) aes(partial) idea(int) 
blowfish(idx)
compiler: information not available
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes256 bytes   1024 bytes   8192 bytes
aes-256-gcm  47427.78k50805.69k51347.19k52207.47k51858.50k

# openssl speed -evp aes-256-gcm
Doing aes-256-gcm for 3s on 16 size blocks: 32056370 aes-256-gcm's in 3.01s
Doing aes-256-gcm for 3s on 64 size blocks: 21569563 aes-256-gcm's in 3.01s
Doing aes-256-gcm for 3s on 256 size blocks: 8523369 aes-256-gcm's in 3.00s
Doing aes-256-gcm for 3s on 1024 size blocks: 2528081 aes-256-gcm's in 3.01s
Doing aes-256-gcm for 3s on 8192 size blocks: 334502 aes-256-gcm's in 3.01s
Doing aes-256-gcm for 3s on 16384 size blocks: 167762 aes-256-gcm's in 3.02s
OpenSSL 1.1.1d-freebsd  10 Sep 2019
built on: 

Re: Odd /tmp behavior

[Jordan Geoghegan]

On 2020-01-07 09:16, Raymond, David wrote:

On an AMD-64 workstation /tmp fills up to 105% according to df,
apparently as a result of UNIX pipes in a shell script passing a whole
lot of moderately big files. Examination of /tmp with du and ls -gal
on /tmp shows no big files and trying to delete everything that is
there has no effect.  Rebooting cleans out /tmp.

I had /tmp mounted with the standard options + softdep.  I eliminated
softdep and the problem appears to have gone away.

Any ideas on what is going on with softdep here?  Dmesg shows a long
series of "/tmp file system full" messages.

Dave Raymond



Using softdep on /tmp is a silly idea. I see way too many people who 
don't understand how softdep works.

Re: Traffic prioritization inside VPN

[Jordan Geoghegan]

On 2020-01-02 11:13, Radek wrote:

what about working directly on rsync side, specifying the maximum
transfer rate? (--bwlimit option)

Setting the hard transfer rate/limit on the rsync side is not what I need. I 
want my boxes to be able to use whole available bandwidth anytime. I mean if 
other services need some bandwitdh they just get it with higher priority and my 
boxes always can use *the rest*. If there is a quiet it the network my boxes 
can use the whole highway.

On Thu, 2 Jan 2020 17:57:19 +0100
fRANz  wrote:


On Thu, Jan 2, 2020 at 3:51 PM radek  wrote:


I tried to do it by "catching" this traffic on [fw_rac]/[fw_krz] by specific 
rules [1] and setting the lowest priority fot it.
Unfortunately it doesn't seem to work as expected. Bandwidth seems to be shared 
roughly equally with other traffic (tested with pushing data (netcat) through 
VPN in the same time).
I would appreciate your advice or any clues on what I have done wrong. Thank 
you.

what about working directly on rsync side, specifying the maximum
transfer rate? (--bwlimit option)
-f





FQ-CoDel may be useful for ensuring fair sharing of bandwidth. I use it 
on some bandwidth constrained links and it works quite nicely at 
ensuring that bandwidth is fairly distributed. I've found that adjusting 
the quantum level to around 300 is nice for ensuring interactive 
applications remain responsive/low latency, by prioritizing smaller 
packets over larger bulk transfer packets.

Re: OpenBSD pf - redirect all DNS queries to local DNS server

[Jordan Geoghegan]

On 2019-12-17 13:24, jin wrote:

Sorry for top posting.

Looks like you need an ip address lists which is updated dynamically. But
this method is not like what you described here. It doesn't response back
an IP address but it does block requests which is trying to get those
ad.servers. if you wish you can choose action to "reject" instead of
"drop". I choose to reject requests because of we just have two devices at
home network.

There are couple of sites which shares ad server names and ip addresses as
a list and they update those lists. As described in below link, you can use
a script to stop traffic which you don't want to have. Basically the script
updates 'source of bad ad server list' periodically and feed your pf rules.

https://www.geoghegan.ca/pfbadhost.html

On Tue, Dec 17, 2019, 23:57 lu hu  wrote:


Our little home network:

ISP -> ROUTER -> SWITCH -> WIFI APs -> CLIENTS

ROUTER: OpenBSD 6.5, giving DHCP+fwing internet to the WIFI APs. Based on
https://www.openbsd.org/faq/pf/example1.html#pf and
https://www.openbsd.org/faq/pf/example1.html#dhcp

CLIENTS: laptops, smartphones.

So everything is going through the ROUTER.

We can see a https://www.openbsd.org/faq/pf/example1.html#dns DOC for how
to setup a DNS server, ~ok.

AD filtering. We would like to have one, but not a fancy one, just a
working one.

Based on "bad hosts", ex.: if a client queries iamAD.foo, then answer it
back as 127.0.0.1, so the clients will try to connect to themselfes, which
will end up not showing the AD.

The big question: Is there any DOC for OpenBSD about this? What pf rules
needed to redirect any DNS server (ex.: 8.8.8.8 or 1.1.1.1) requests to the
DNS server running on the ROUTER, coming from the CLIENTS?

So ex.: if a smartphone CLIENT wants to query iamAD.foo domain to get ADs,
it will only get back 127.0.0.1





Hey, I'm the author of that script. If you're looking to block ads via 
DNS, geoghegan.ca/unbound-adblock.html may be what you're looking for.


It pulls a popular ad server blocklist and makes unbound return NXDOMAIN 
when a device tries to query a known ad server. Certain devices have 
issues when redirecting their querys to 127.0.0.1 or 0.0.0.0, and some 
devices may waste time retrying queries for a long period of time. 
Setting static redirects to a particular address causes unbound to eat 
up a ton of memory, wheras returning NXDomain uses almost no memory.


Cheers,

Jordan

Re: unbound network optimizations

[Jordan Geoghegan]

On 2019-12-12 06:21, Winter Paulson wrote:

Hello,

I'm also experiencing the "Host is down" problem:

unbound: [85343:0] error: recvfrom 361 failed: Host is down

Running openbsd 6.6 (GENERIC.MP), current syspatch,
native unbound as a full resolver, pf disabled.

OpenBSD is a guest VM on a debian buster host using virtual e1000
network card ("Intel 82540EM" driver in openbsd). No firewall
in between. The VM is a tor-exit node.


I've heard others recommend using the vio driver over the em driver 
numerous times on here if running a virtualized instance. You may have a 
better time than you are now by using the VirtIO drivers. The intel nic 
emulation can sometimes have issues. Better to use an interface designed 
for virtualized environments.



Cheers,

Jordan

Re: Can't select files to upload in a browsers

[Jordan Geoghegan]

On 2019-12-06 02:56, dmitry.sensei wrote:

Firefox and Chromium browser, in the file selection window for upload,
does not show the contents of directories other than the Downloads
directory


OpenBSD 6.6-current
OpenBSD 6.6 GENERIC.MP#509 amd64

openbox-3.6.1p7 small, fast & usable window manager
firefox-71.0Mozilla web browser
chromium-78.0.3904.106 Chromium browser
gtk+2-2.24.32p8 multi-platform graphical toolkit
gtk+3-3.24.13   multi-platform graphical toolkit
gtk+4-3.96.0p6  multi-platform graphical toolkit



That's because both browsers are using unveil.

Re: sysupgrade to 6.6 failed at comp66.tgz

[Jordan Geoghegan]

On 2019-11-23 14:20, Rachel Roch wrote:



This topic has been beat to death. deraadt@ and other have made it clear that 
if you do not install all the sets, you are running an unsupported 
configuration. It has been stated that if people keep bitching, they're just 
going to merge the release sets into one set.

I like the fact that there are separate sets. A number of times I've had to 
squeeze an install onto a <2GB disk, and it was useful being able to select 
only the specific sets I wanted/needed, while at the same time acknowledging that 
it was indeed an unsupported configuration.

If people are going to try and be edgelords by refusing to install all the 
sets, then it's up to them to maintain and diagnose their unsupported 
configuration.


You can't seriously be calling "-x* -game*" an unsupported configuration ?  
Seems to me like a sensible thing to do on any box that's going to be headless for its 
entire life and only ever accessed via SSH (or text console at a push).



Running without the X sets is most certainly an unsupported 
configuration. Many packages/ports have dependencies on things in the X 
sets, I've been bit by this issue my self. Certain ports have "no_x11" 
flavours, but it's not a guarantee.


With regards to the game sets, they're less than 5MB, so its pretty 
irrelevant.


Forgoing the X sets does nothing for security, and ultimately further 
removes you from a standard supported install. Unless you're trying to 
do an install on a super small disk, just install all the sets. If 
you're running any sort of modern or important production machine, 
you're going to have more than enough disk space to install all the sets.


If you want to be an edgelord and not install all the sets, then by all 
means, please enjoy your unsupported system. Don't come back bitching 
and moaning if something breaks.

Re: sysupgrade to 6.6 failed at comp66.tgz

[Jordan Geoghegan]

On 2019-11-23 13:45, Rachel Roch wrote:



- maybe sysupgrade needs to be patched to avoid this issue?


Probably not. sysupgrade has assumptions baked in to it which have
evidently been rendered invalid either by another tool or by the
person using them. That tool is where the patch most likely ought
to be directed.




If I may make a little comment here.

Surely it is a little bit questionable to "bake assumptions" into sysupgrade 
that everybody is going to do a complete install when the OpenBSD installer itself gives 
you the option to select what is going to be installed.

At the very least, may I suggest that even if the developers don't want to increase 
the intelligence of sysupgrade that they at least code in some sanity checks (e.g. 
"pick a file - or two - at random from the core tgz files that you would 
normally expect to be present on the system if a 'full-default' install was done.  
If file not present, then throw a horrid error message and abort).

It strikes me as a little silly to put a tool out there that you know will trash (or at 
least severely brick) a user's system just because of some severely opinionated 
"baked assumptions" coded into it.



This topic has been beat to death. deraadt@ and other have made it clear 
that if you do not install all the sets, you are running an unsupported 
configuration. It has been stated that if people keep bitching, they're 
just going to merge the release sets into one set.


I like the fact that there are separate sets. A number of times I've had 
to squeeze an install onto a <2GB disk, and it was useful being able to 
select only the specific sets I wanted/needed, while at the same time 
acknowledging that it was indeed an unsupported configuration.


If people are going to try and be edgelords by refusing to install all 
the sets, then it's up to them to maintain and diagnose their 
unsupported configuration.

Re: Running Windows inside vmm/vmd VM.

[Jordan Geoghegan]

On 2019-11-22 19:53, Jes wrote:

On Fri Nov 22, 2019 at 9:04 PM Dumitru Moldovan wrote:

  Supported guest operating systems are currently limited to OpenBSD and
  Linux. As there is no VGA support yet, the guest OS must support serial
  console.

Speaking of this, does anyone here have any experience running Linux VMs
on vmm/vmd? I threw Alpine/Debian installs together recently and
they seemed to work well. Looking for anyone with longer-term experience
as I'm interested in setting up a VPS hosting service on vmm/vmd, and
would appreciate any advice or anecdotes.

I have some Alpine and Void Linux installs running on vmm. They work 
well, with some caveats.


You may have issues with your VMs clocks. OpenBSD guests in vmm are now 
able to use the pvclock driver, which has greatly improved time keeping 
on my VMs, although I still do have some erratic clock jumping, but at 
least it's not so bad that ntpd can't keep up with it.


However the timekeeping situation for my Linux VMs is bleak. On both 
Void and Alpine, no clocks are even detected. In the dmesg it complains 
about the TSC clock source being unstable. Ultimately, we're left with 
only jiffies as a clock source option:


void$ cat 
/sys/devices/system/clocksource/clocksource0/available_clocksource

refined-jiffies jiffies

As a result, my clocks run at about one third of real time.

I've tried the Linux VM's on both an old Xeon machine as well as a 
modern Ryzen machine, and the clock situation seems to be equally bad on 
both of them.


...

Clock issues aside, I've found Linux guests to get better networking 
throughput on vmm than OpenBSD guests.


A few results from benchmarking Alpine vs Void vs OBSD, with iperf3:

(vmm host is older xeon rig, iperf3 tester is ryzen desktop)

Alpine got this result:

[ ID] Interval   Transfer Bitrate
[  5]   0.00-10.00  sec   511 MBytes   429 Mbits/sec  sender
[  5]   0.00-8.60   sec   511 MBytes   499 Mbits/sec  
receiver


Void Linux Got this result:

[ ID] Interval   Transfer Bitrate
[  5]   0.00-10.00  sec   611 MBytes   512 Mbits/sec  sender
[  5]   0.00-7.00   sec   610 MBytes   732 Mbits/sec  
receiver


And OpenBSD got this result:

[ ID] Interval   Transfer Bitrate
[  5]   0.00-10.00  sec   299 MBytes   251 Mbits/sec  sender
[  5]   0.00-10.19  sec   299 MBytes   246 Mbits/sec  
receiver


Because folks always freak out when tcpbench is forgotten about, I 
tested tcpbench as well between the two machines running OpenBSD:


Peak Mbps:  231.240 Avg Mbps:  204.423

I know that was some very unscientific testing, but hey, you asked for 
anecdotes.


Cheers,

Jordan