[patch] neomutt: enable pcre2 to parse non-ASCII urls

[James Cook]

Hi,

neomutt can't properly parse non-ASCII URLs on OpenBSD without
pcre2. It doesn't affect me personally but I imagine someone out
there will be unhappy that neomutt can't open their IMAP folder.
See:

https://github.com/neomutt/neomutt/issues/4352

You can see it in neomutt by pressing "c" for the "change-folder"
command, then entering this URL from neomutt's test_url_parse(): 

imaps://foo...@gmail.com@imap.gmail.com/Отправленные

Currently, URL parsing fails and neomutt just silently takes you
to an empty index showing "(no mailbox)". With the below patch it
shows some login failures first. (I don't have any real non-ASCII
mailboxes to test it out on.)

I also removed the --idn=0 configure option. It seems to have become
a no-op in commit 707e03e3.

-- 
James


diff /usr/ports
commit - 69e51b5b54a5eaf011a238d8566e08ade3de75eb
path + /usr/ports
blob - 10a691427f6899573b95320b0caf7aabfbfbd127
file + mail/neomutt/Makefile
--- mail/neomutt/Makefile
+++ mail/neomutt/Makefile
@@ -3,6 +3,7 @@ COMMENT=tty-based e-mail client, forked from Mutt
 GH_ACCOUNT=neomutt
 GH_PROJECT=neomutt
 GH_TAGNAME=20240425
+REVISION=  0
 
 CATEGORIES=mail
 HOMEPAGE=  https://neomutt.org/
@@ -15,11 +16,12 @@ PERMIT_PACKAGE= Yes
 COMPILER=  base-clang ports-gcc
 
 # uses pledge()
-WANTLIB+=  c crypto curses iconv idn2 intl ssl tdb z
+WANTLIB+=  c crypto curses iconv idn2 intl pcre2-8 ssl tdb z
 
 LIB_DEPENDS+=  databases/tdb \
devel/gettext,-runtime \
-   devel/libidn2
+   devel/libidn2 \
+   devel/pcre2
 BUILD_DEPENDS+=devel/gettext,-tools \
lang/jimtcl \
textproc/docbook-xsl \
@@ -35,9 +37,9 @@ CONFIGURE_ARGS+= --mandir="${PREFIX}/man" \
--with-ui=ncurses \
--docdir="${PREFIX}/share/doc/neomutt" \
--with-lock=flock \
-   --idn=0 \
--idn2 \
--fmemopen \
+   --pcre2 \
--ssl \
--tdb \
--debug

Re: [maintainer update] sysutils/fzf 0.52.1

[James Cook]

On Tue, May 21, 2024 at 12:17:50PM GMT, Laurent Cheylus wrote:

Hi,

a simple update for sysutils/fzf version 0.52.1
Changelog for version 0.52.0 (0.52.1 is for a Windows bugfix only): 
https://github.com/junegunn/fzf/releases/tag/0.52.0

- update Makefile, modules.inc and distinfo for this version

$ go list -m github.com/junegunn/fzf@0.52.1
github.com/junegunn/fzf v0.0.0-20240513165430-6432f00f0d02

Build and tests OK on current/amd64 with Go version 1.22.3

Please tests welcome and commit if OK.

Laurent


Thanks. Just installed it. It's working well so far with my scripts 
that use fzf, and I like the new --highlight-line option.


--
James

Re: [new] gurk-rs - a cli signal client

[James Cook]

On Thu, May 16, 2024 at 07:53:41PM GMT, Stefan Hagen wrote:

James Cook wrote (2024-05-14 21:21 CEST):

> I wasn't able to link to my phone because the QR code is not displayed in a
> usable way.  Here are the first three lines of what I assume is supposed to
> be a QR code.
>
>$ gurk
>Linking new device with device name: gurk@angel
>   ▄▄▄ ▄ ▄   ▄   
▄▄ ▄ ▄  ▄  ▄  ▄  ▄ ▄ ▄ ▄
> ▄ ▄ ▄  ▄▄▄▄ ▄    ▄▄ ▄▄   ▄▄▄

Sorry, format=flowed messed that up. Here are the first three lines
again:

   ▄▄▄ ▄ ▄   ▄   ▄▄ ▄ ▄  ▄
▄  ▄  ▄ ▄ ▄ ▄ ▄ ▄ ▄
▄▄▄▄ ▄    ▄▄ ▄▄   ▄▄▄


Try playing with a different font / different terminal.
It should look like a square'ish block that resembles
a QR code.

Best regards,
Stefan


Some more info:

I tried redirecting gurk's stdout to a file. Feeding that output to vis, it 
begins like this:

Linking new device with device name: gurk@angel
\^[[m\^[[m \^[[m\^[[m\^[[m\^[[m \^[[m\^[[m\^[[m\^[[m \^[[m\^[[m

This seems broken, because when I wrote my own program to call 
qr2term::print_qr, the output began like this (again, viewed with

vis):

\^[[48;5;15m\^[[38;5;0m \^[[49m\^[[39m\^[[48;5;15m

It looks like some terminal escape codes are not properly generated 
when gurk calls qr2term::print_qr. My own program's output is a 
usable QR code.


I tried gurk in xterm, in tmux in xterm, and in gnome-terminal 
(hastily installed just for this purpose; gnome-terminal is using 
a weird ugly font and I didn't dig into why). In all three cases, 
gurk|vis|head shows the same (as far as I can see) sequence of 
\^[[m\^[m The TERM environment variable is "xterm", "screen"

and "xterm-256color respectively in those three cases.




Details on how I manually tried qr2term::print_qr:

I built and ran the following program:

use qr2term;

fn main() {
qr2term::print_qr("https://rust-lang.org/;).unwrap();
}

and it works fine: I see a (kind of ugly) QR code on the terminal 
that I can scan. I built it with the following Cargo.toml file, 
which I generated with "cargo init" then "cargo add qr2term":


[package]
name = "r"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at 
https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
qr2term = "0.3.1"

--
James

Re: [new] gurk-rs - a cli signal client

[James Cook]

> I wasn't able to link to my phone because the QR code is not displayed in a
> usable way.  Here are the first three lines of what I assume is supposed to
> be a QR code.
> 
>   $ gurk
>   Linking new device with device name: gurk@angel
>  ▄▄▄ ▄ ▄   ▄  
>  ▄▄ ▄ ▄  ▄  ▄  ▄  ▄ ▄ ▄ ▄
> ▄ ▄ ▄ ▄▄▄▄ ▄    ▄▄ ▄▄   ▄▄▄

Sorry, format=flowed messed that up. Here are the first three lines
again:

   ▄▄▄ ▄ ▄   ▄   ▄▄ ▄ ▄  ▄
▄  ▄  ▄ ▄ ▄ ▄ ▄ ▄ ▄
▄▄▄▄ ▄    ▄▄ ▄▄   ▄▄▄

-- 
James

Re: [new] gurk-rs - a cli signal client

[James Cook]

On Tue, Apr 16, 2024 at 04:52:46PM GMT, Edd Barrett wrote:

On Sat, Mar 23, 2024 at 09:56:07PM +0100, Stefan Hagen wrote:

I've updated gurk-rs on openbsd-wip to version 0.4.3. It seems to start
more reliable now. I could exchange a few messages. Once a message got
deleted, further messages could not be sent from gurk.


I just tried this and it worked until someone sent me a message, then it
dropped me back to the shall and spammed the screen with weird characters.

Such a shame, I could really use a client like this.

--
Best Regards
Edd Barrett

https://www.theunixzoo.co.uk



I just tried this and wasn't able to link to my phone.

I had to patch the Makefile slightly to get it to build; see [0].

I wasn't able to link to my phone because the QR code is not displayed 
in a usable way.  Here are the first three lines of what I assume is 
supposed to be a QR code.


$ gurk
Linking new device with device name: gurk@angel
		 
	   ▄▄▄ ▄ ▄   ▄   ▄▄ ▄ ▄  ▄   
		▄  ▄  ▄ ▄ ▄ ▄ ▄ ▄ ▄  
	▄▄▄▄ ▄    ▄▄ ▄▄   ▄▄▄  

The whole thing seems to be made out of spaces and U+2584 characters. 
The Signal app does not recognize it as a QR code. Has anyone else 
encountered this behaviour?


Just in case, I tried setting LANG=en_US.UTF-8, and then LANG=C, 
and then instead LC_CTYPE=en_US.UTF-8, but none of those helped.


--
James

[0] https://github.com/jasperla/openbsd-wip/pull/175

Re: [update] editors/vis 0.8 -> 0.9

[James Cook]

> > May I ask how are you running the tests? Just 'make test' from the ports
> > directory as usual? Additionally, does the processes started by the
> > test.sh script not inherit the shell it is executed with?
> 
> Yes, "make test" from the ports dir. Might have been "make 
> FETCH_PACKAGES=-Dsnap test".
> 
> I just tried again with your new patch:
>   make clean
>   make
>   make install  (give doas my password)
>   make test
> Same result, as far as I can tell. I attached vis-0.9/test.log from
> my WRKOBJDIR.
> 
> My /usr/ports is managed by got, and is currently at commit 419d15f5,
> from May 3 at 16:00 UTC.
> 
> > $ cat /usr/ports/pobj/vis-0.9/vis-0.9/test/sam/test.sh | head -1
> > #!/bin/sh
> 
> Yes (but my WRKOBJDIR is /home/pobj):
> $ cat /home/pobj/vis-0.9/vis-0.9/test/sam/test.sh | head -1
> #!/bin/sh

A bit more info:

I get the same result on my laptop. (The bug I mentioned in my
previous email was a broken version of dwz; pkg_add -u dwz fixed
it.)

If I print the value of SHELL in sam/test.sh, or in the test rule
of vis's top-level make-file, I see an empty string.

That is: I edit /usr/ports/pobj/vis-0.9/vis-0.9/test/Makefile so it begins with:

test:
sh -c 'echo shell _$${SHELL}_'
@$(MAKE) -C core
@$(MAKE) -C lua
@$(MAKE) -C vis
@$(MAKE) -C sam
@$(MAKE) -C vim

and then when I run "make test" under /usr/ports/editors/vis the beginning of 
the output is:

$ make test
===>  Regression tests for vis-0.9
[ -e test/Makefile ] || gmake test-update
gmake[1]: Entering directory '/usr/ports/pobj/vis-0.9/vis-0.9/test'
sh -c 'echo shell _${SHELL}_'
shell __

(As far as I can tell, putting #!/bin/sh at the start of a script
doesn't affect the value of SHELL when the script is run.)

make show=SHELL still shows /bin/sh on this machine.

-- 
James

Re: [update] editors/vis 0.8 -> 0.9

[James Cook]

On Sun, May 05, 2024 at 11:48:47AM GMT, Johannes Thyssen Tishman wrote:
> 2024-05-03T19:20:24Z "Johannes Thyssen Tishman" :
> > 2024-05-03T19:06:51Z James Cook :
> >
> > > As far as I can tell, the problem is that SHELL is /sbin/nologin
> > > for user _pbuild. (I have PORTS_PRIVSEP=Yes in /etc/mk.conf.)
> >
> > Hmm weird, I also use PORTS_PRIVSEP and had no issues with the tests.
> > I'll have to take another look at it this weekend.
> >
> > I'll send a revision anyways because I missed that we can probably
> > drop the post-extract rule by specifying the subdir with DIST_TUPLE.
> 
> @James, I was able to reproduce the error you've found by running the
> commands you provided with and without SHELL. However, tests are still
> passing for me (see attached test.log). I would expect that SHELL is set
> by the building environment no? E.g.:
> 
> $ make show=SHELL
> /bin/sh

I see this too:
$ make show=SHELL
/bin/sh

> May I ask how are you running the tests? Just 'make test' from the ports
> directory as usual? Additionally, does the processes started by the
> test.sh script not inherit the shell it is executed with?

Yes, "make test" from the ports dir. Might have been "make 
FETCH_PACKAGES=-Dsnap test".

I just tried again with your new patch:
make clean
make
make install  (give doas my password)
make test
Same result, as far as I can tell. I attached vis-0.9/test.log from
my WRKOBJDIR.

My /usr/ports is managed by got, and is currently at commit 419d15f5,
from May 3 at 16:00 UTC.

> $ cat /usr/ports/pobj/vis-0.9/vis-0.9/test/sam/test.sh | head -1
> #!/bin/sh

Yes (but my WRKOBJDIR is /home/pobj):
$ cat /home/pobj/vis-0.9/vis-0.9/test/sam/test.sh | head -1
#!/bin/sh

> For more context, here is what I have in my /etc/mk.conf:
> 
> $ cat /etc/mk.conf
> PORTS_PRIVSEP=yes
> SUDO=doas
> PORTSDIR_PATH=${PORTSDIR}/mystuff:${PORTSDIR}/wip:${PORTSDIR}
> USE_CCACHE=yes
> FETCH_PACKAGES=no

$ cat /etc/mk.conf  
  
PORTS_PRIVSEP=Yes
WRKOBJDIR=/home/pobj

I just tried to reproduce on another computer (laptop) but it's
running into some weird "Abort trap (core dump)" error on "make
install" that I don't have time to debug right now.
> Extracting debug info from 
/usr/ports/pobj/vis-0.9/fake-amd64/usr/local/bin/vis-digraph
Abort trap (core dumped)
I noticed the same error when "make FETCH_PACKAGES=-Dsnap
test" tried to install gettext from source. I installed gettext-runtime,
gettext-tools with pkg_add instead and crossed my fingers; probably
that's somehow related.

For the next few days I will only have that laptop but if I find
time I will try to see if I can reproduce, and dig deeper if so.

-- 
James
[ -e test/Makefile ] || gmake test-update
gmake[1]: Entering directory '/home/pobj/vis-0.9/vis-0.9/test'
gmake[2]: Entering directory '/home/pobj/vis-0.9/vis-0.9/test/core'
Generating ccan configuration header
ccan-config.c(/tmp/ccan-config-213e92.o:(connect_args)): warning: strcpy() is 
almost always misused, please use strlcpy()
ccan-config.c(/tmp/ccan-config-213e92.o:(run_test)): warning: sprintf() is 
often misused, please use snprintf()
Compiling buffer-test binary
Compiling map-test binary
Compiling array-test binary
Compiling text-test binary
ok 1 - Initialization
ok 2 - Insert zero length data
ok 3 - Insert string at invalid position
ok 4 - Insert empty string
ok 5 - Insert string at start
ok 6 - Insert string in middle
ok 7 - Insert string at end
ok 8 - Put empty string
ok 9 - Put string
ok 10 - Prepend string
ok 11 - Append string
ok 12 - Release
ok 13 - Insert zero length data
ok 14 - Insert data at start
ok 15 - Insert data in middle
ok 16 - Insert data at end
ok 17 - Clear
ok 18 - Put zero length data
ok 19 - Put data
ok 20 - Prepend data
ok 21 - Append data
ok 22 - Grow
ok 23 - Move
ok 24 - Modify after move
ok 25 - Set formatted
ok 26 - Set formatted overwrite
ok 27 - Set formatted empty string
ok 28 - Append formatted
ok 29 - Append formatted empty string
1..29
ok 1 - Creation
ok 2 - First on empty map
ok 3 - Empty prefix map
ok 4 - Get non-existing key
ok 5 - Contains non-existing key
ok 6 - Closest non-existing key
ok 7 - Put NULL value
ok 8 - Put 1
ok 9 - First on map with 1 value
ok 10 - First on prefix map
ok 11 - Contains existing key
ok 12 - Closest match existing key
ok 13 - Put duplicate
ok 14 - Put 2
ok 15 - Put 3
ok 16 - First on prefix map with multiple suffixes
ok 17 - Copy
ok 18 - Not empty after copying
ok 19 - Compare map content
ok 20 - Compare map content
ok 21 - Compare map content
ok 22 - Compare map content
ok 23 - Compare map content
ok 24 - Compare map content
ok 25 - Iterate stop condition
ok 26 - Closest ambigious
ok 27 - Iterate map
ok 28 

Re: [update] editors/vis 0.8 -> 0.9

[James Cook]

On Fri, May 03, 2024 at 07:06:45PM GMT, James Cook wrote:
> On Fri, May 03, 2024 at 12:26:15PM GMT, Johannes Thyssen Tishman wrote:
> > Please find below an update for editors/vis to version 0.9. The tests
> > (which are hosted separately) didn't seem to get an update, but they all
> > still pass.
> > 
> > Cc'd maintainer.
> 
> Thank you, this seems to work so far, but "make test" fails for me.
> 
> As far as I can tell, the problem is that SHELL is /sbin/nologin
> for user _pbuild. (I have PORTS_PRIVSEP=Yes in /etc/mk.conf.)
> For example, the following command causes vis to correctly capitalize
> the test file's content; without SHELL=/bin/sh the vis status bar
> shows "Command failed" and the text in the file is not capitalized:
> 
>   doas -u _pbuild sh -c 'SHELL=/bin/sh VIS_PATH=. ../../vis 
> commands/filter-capitalize.in'

Oops, forgot to specify that I'm running that in the directory
vis-0.9/vis-0.9/test/sam under my pobj directory.

-- 
James

Re: [update] editors/vis 0.8 -> 0.9

[James Cook]

On Fri, May 03, 2024 at 12:26:15PM GMT, Johannes Thyssen Tishman wrote:
> Please find below an update for editors/vis to version 0.9. The tests
> (which are hosted separately) didn't seem to get an update, but they all
> still pass.
> 
> Cc'd maintainer.

Thank you, this seems to work so far, but "make test" fails for me.

As far as I can tell, the problem is that SHELL is /sbin/nologin
for user _pbuild. (I have PORTS_PRIVSEP=Yes in /etc/mk.conf.)
For example, the following command causes vis to correctly capitalize
the test file's content; without SHELL=/bin/sh the vis status bar
shows "Command failed" and the text in the file is not capitalized:

doas -u _pbuild sh -c 'SHELL=/bin/sh VIS_PATH=. ../../vis 
commands/filter-capitalize.in'

I pasted the end of the output of "make test" below my signature.

-- 
James


gmake[2]: Entering directory '/home/pobj/vis-0.9/vis-0.9/test/sam'
/usr/local/plan9/bin/sam
vis 0.9 +curses +lua +tre
Running test ./addresses/columns with sam ... OK
Running test ./addresses/columns with vis ... OK
Running test ./addresses/first-last with sam ... OK
Running test ./addresses/first-last with vis ... OK
Running test ./addresses/lines with sam ... OK
Running test ./addresses/lines with vis ... OK
Running test ./addresses/second with sam ... OK
Running test ./addresses/second with vis ... OK
Running test ./commands/filter-capitalize with sam ... OK
Running test ./commands/filter-capitalize with vis ... FAIL
Running test ./commands/group with sam ... OK
Running test ./commands/group with vis ... OK
Running test ./commands/long-text with sam ... OK
Running test ./commands/long-text with vis ... OK
Running test ./commands/loop-lines1 with sam ... OK
Running test ./commands/loop-lines1 with vis ... OK
Running test ./commands/loop-lines2 with sam ... OK
Running test ./commands/loop-lines2 with vis ... OK
Running test ./commands/loop-lines3 with sam ... OK
Running test ./commands/loop-lines3 with vis ... OK
Running test ./commands/loop-lines4 with sam ... OK
Running test ./commands/loop-lines4 with vis ... OK
Running test ./commands/loop-lines5 with sam ... OK
Running test ./commands/loop-lines5 with vis ... OK
Running test ./commands/loop-lines6 with sam ... OK
Running test ./commands/loop-lines6 with vis ... OK
Running test ./commands/loop-lines7 with sam ... OK
Running test ./commands/loop-lines7 with vis ... OK
Running test ./commands/loop-lines8 with sam ... OK
Running test ./commands/loop-lines8 with vis ... OK
Running test ./commands/loop-lines9 with sam ... OK
Running test ./commands/loop-lines9 with vis ... OK
Running test ./commands/pipe-in with sam ... OK
Running test ./commands/pipe-in with vis ... FAIL
Running test ./commands/pipe-out with sam ... OK
Running test ./commands/pipe-out with vis ... FAIL
Running test ./commands/repeated-shell with sam ... OK
Running test ./commands/repeated-shell with vis ... FAIL
Running test ./commands/unicode-replace with sam ... OK
Running test ./commands/unicode-replace with vis ... OK
Running test ./errors/conflict with sam ... OK
Running test ./errors/conflict with vis ... OK
Running test ./errors/read with sam ... OK
Running test ./errors/read with vis ... OK
Running test ./errors/unbalanced-group with sam ... OK
Running test ./errors/unbalanced-group with vis ... OK
Running test ./examples/comment-functions with sam ... OK
Running test ./examples/comment-functions with vis ... OK
Running test ./examples/delete-empty-lines with sam ... OK
Running test ./examples/delete-empty-lines with vis ... OK
Running test ./examples/swap-words with sam ... OK
Running test ./examples/swap-words with vis ... OK
Tests ok 22/26
gmake[2]: *** [Makefile:2: test] Error 1
gmake[2]: Leaving directory '/home/pobj/vis-0.9/vis-0.9/test/sam'
gmake[1]: *** [Makefile:5: test] Error 2
gmake[1]: Leaving directory '/home/pobj/vis-0.9/vis-0.9/test'
gmake: *** [Makefile:156: test] Error 2
*** Error 2 in . (/usr/ports/infrastructure/mk/bsd.port.mk:3096 
'/home/pobj/vis-0.9/.test_done': @cd /home/pobj/vis-0.9/vis-0.9 && exec 3>&1...)
*** Error 2 in /usr/ports/editors/vis 
(/usr/ports/infrastructure/mk/bsd.port.mk:2704 'test': @lock=vis-0.9;  export 
_LOCKS_HELD=" vis-0.9"; ...)

Re: -current Haskell ports aborting with SIGILL

[James Cook]

On Mon, Apr 22, 2024 at 11:21:36AM GMT, Greg Steuck wrote:
> Stuart Henderson  writes:
> 
> > On 2024/04/22 10:30, Greg Steuck wrote:
> >> > If it would help, I could update my old AMD machine to -current
> >> > and check ghc works with the two patches removed, once I've finished
> >> > trying out the patch I just sent for 7.5.
> >> 
> >> Thanks James for working through this. Yes, we need the new development
> >> to happen on -current ports with -current base system. We'd also want a
> >> more complicated patch than the one you just sent because base supports
> >> avx-512 now.
> >
> > We probably don't need any patches for this in -current now that avx-512
> > opcodes are supported by the OS.
> 
> James, if you'd like to play with this on -current, please remove both
> patch-libraries_text_simdutf_simdutf_h and
> patch-libraries_text_cbits_measure_off_c
> 
> This should make the offending check disappear and the pre-existing
> checks should correctly report the OS doesn't support these instructions
> on your machine. We can then confirm that avx-512 is working for people
> who previously had problems.
> 
> Thanks
> Greg

It works. I upgraded to -current and built ghc (now version 9.6.5)
after deleting patches/patch-libraries_text_cbits_measure_off_c and
patches/patch-libraries_text_simdutf_simdutf_h. All three examples
now work (i.e. no SIGILL):

- T.take 1 $ T.pack "aa" in ghci
- pandoc 

Re: -current Haskell ports aborting with SIGILL

[James Cook]

On Sun, Apr 21, 2024 at 09:03:30PM +, James Cook wrote:
> On Sun, Apr 21, 2024 at 11:49:46AM -0700, Greg Steuck wrote:
> > Stuart Henderson  writes:
> > 
> > > This is in the avx512 checks in the text library again, I think it must
> > > be patch-libraries_text_cbits_measure_off_c (the simdutf one doesn't
> > > explicitly check for xgetbv but it does check for osxsave so I think
> > > wouldn't have executed the xgetbv opcode on this cpu).
> > >
> > > As -current does now have avx512 support in the kernel we probably
> > > should be able to remove that patch, but it needs testing on an avx512
> > > machine as well as that old Phenom.
> > 
> > Sadly I have neither nearby. Furthermore, sumdutf upstream doesn't have
> > a fix for this issue either. So this will have to be original work.
> > 
> > https://github.com/simdutf/simdutf/blob/master/include/simdutf/internal/isadetection.h#L232
> > 
> > Thanks
> > Greg
> 
> The following patch fixes at least the ghci example on my AMD machine:
> 
>   $ ghci  
>   GHCi, version 9.6.4: https://www.haskell.org/ghc/  :? for help
>   ghci> import qualified Data.Text as T
>   ghci> T.take 1 $ T.pack "aa"
>   "a"

Confirmed that the following are also fixed, which were broken
before for me:
- cabal-bundler (I just invoked it with no arguments)
- building the pandoc port
- pandoc /dev/null

-- 
James

Re: -current Haskell ports aborting with SIGILL

[James Cook]

On Sun, Apr 21, 2024 at 11:49:46AM -0700, Greg Steuck wrote:
> Stuart Henderson  writes:
> 
> > This is in the avx512 checks in the text library again, I think it must
> > be patch-libraries_text_cbits_measure_off_c (the simdutf one doesn't
> > explicitly check for xgetbv but it does check for osxsave so I think
> > wouldn't have executed the xgetbv opcode on this cpu).
> >
> > As -current does now have avx512 support in the kernel we probably
> > should be able to remove that patch, but it needs testing on an avx512
> > machine as well as that old Phenom.
> 
> Sadly I have neither nearby. Furthermore, sumdutf upstream doesn't have
> a fix for this issue either. So this will have to be original work.
> 
> https://github.com/simdutf/simdutf/blob/master/include/simdutf/internal/isadetection.h#L232
> 
> Thanks
> Greg

The line you linked to comes after a check for cpuid_bit::osxsave,
so I don't think it would get reached on machines that don't have
xgetbv, i.e. it should be fine. Similarly, the existing
patch-libraries_text_simdutf_simdutf_h in ports doesn't seem to
cause a problem with my AMD machine. At least, now I have pandoc
working; not sure if I actually exercised that code in simdutf.h.

If it would help, I could update my old AMD machine to -current
and check ghc works with the two patches removed, once I've finished
trying out the patch I just sent for 7.5.

-- 
James

Re: -current Haskell ports aborting with SIGILL

[James Cook]

On Sun, Apr 21, 2024 at 11:49:46AM -0700, Greg Steuck wrote:
> Stuart Henderson  writes:
> 
> > This is in the avx512 checks in the text library again, I think it must
> > be patch-libraries_text_cbits_measure_off_c (the simdutf one doesn't
> > explicitly check for xgetbv but it does check for osxsave so I think
> > wouldn't have executed the xgetbv opcode on this cpu).
> >
> > As -current does now have avx512 support in the kernel we probably
> > should be able to remove that patch, but it needs testing on an avx512
> > machine as well as that old Phenom.
> 
> Sadly I have neither nearby. Furthermore, sumdutf upstream doesn't have
> a fix for this issue either. So this will have to be original work.
> 
> https://github.com/simdutf/simdutf/blob/master/include/simdutf/internal/isadetection.h#L232
> 
> Thanks
> Greg

The following patch fixes at least the ghci example on my AMD machine:

$ ghci  
GHCi, version 9.6.4: https://www.haskell.org/ghc/  :? for help
ghci> import qualified Data.Text as T
ghci> T.take 1 $ T.pack "aa"
"a"

It simply patches has_avx512_vl_bw to always return false. Would
this be a good candidate for the stable branch, which doesn't support
AVX-512 anyway?

I will try building pandoc now with the new ghc to see if it's
fixed. (I'm a little fuzzy on how libraries are included... hopefully
pandoc will take the patched library from the updated ghc? Or does
it not work like that?)

-- 
James


Index: Makefile
===
RCS file: /cvs/ports/lang/ghc/Makefile,v
diff -u -p -u -p -r1.223.2.1 Makefile
--- Makefile21 Mar 2024 20:52:54 -  1.223.2.1
+++ Makefile21 Apr 2024 20:59:59 -
@@ -14,7 +14,7 @@ USE_NOEXECONLY =  Yes
 USE_NOBTCFI =  Yes
 
 GHC_VERSION =  9.6.4
-REVISION = 2
+REVISION = 3
 DISTNAME = ghc-${GHC_VERSION}
 CATEGORIES =   lang devel
 HOMEPAGE = https://www.haskell.org/ghc/
Index: patches/patch-libraries_text_cbits_measure_off_c
===
RCS file: /cvs/ports/lang/ghc/patches/patch-libraries_text_cbits_measure_off_c,v
diff -u -p -u -p -r1.2 patch-libraries_text_cbits_measure_off_c
--- patches/patch-libraries_text_cbits_measure_off_c23 Feb 2024 19:45:04 
-  1.2
+++ patches/patch-libraries_text_cbits_measure_off_c21 Apr 2024 20:59:59 
-
@@ -1,23 +1,24 @@
-Don't attempt to use avx512 kernels when the OS doesn't support them
+Don't attempt to use avx512 kernels, because OpenBSD 7.5 doesn't support them
 
 Index: libraries/text/cbits/measure_off.c
 --- libraries/text/cbits/measure_off.c.orig
 +++ libraries/text/cbits/measure_off.c
-@@ -44,12 +44,16 @@
+@@ -42,17 +42,8 @@
+ 
+ #if defined(__x86_64__) && defined(COMPILER_SUPPORTS_AVX512)
  bool has_avx512_vl_bw() {
- #if (__GNUC__ >= 7 || __GNUC__ == 6 && __GNUC_MINOR__ >= 3) || 
defined(__clang_major__)
-   uint32_t eax = 0, ebx = 0, ecx = 0, edx = 0;
-+  uint64_t xcr0;
-   __get_cpuid_count(7, 0, , , , );
-   // https://en.wikipedia.org/wiki/CPUID#EAX=7,_ECX=0:_Extended_Features
-+  __asm__("xgetbv\n\t" : "=a" (xcr0) : "c" (0));
-   const bool has_avx512_bw = ebx & (1 << 30);
-   const bool has_avx512_vl = ebx & (1 << 31);
-+  // XCR0 bits 5, 6, and 7
-+  const bool avx512_os_enabled = (xcr0 & 0xE0) == 0xE0;
-   // printf("cpuid=%d=cpuid\n", has_avx512_bw && has_avx512_vl);
+-#if (__GNUC__ >= 7 || __GNUC__ == 6 && __GNUC_MINOR__ >= 3) || 
defined(__clang_major__)
+-  uint32_t eax = 0, ebx = 0, ecx = 0, edx = 0;
+-  __get_cpuid_count(7, 0, , , , );
+-  // https://en.wikipedia.org/wiki/CPUID#EAX=7,_ECX=0:_Extended_Features
+-  const bool has_avx512_bw = ebx & (1 << 30);
+-  const bool has_avx512_vl = ebx & (1 << 31);
+-  // printf("cpuid=%d=cpuid\n", has_avx512_bw && has_avx512_vl);
 -  return has_avx512_bw && has_avx512_vl;
-+  return has_avx512_bw && has_avx512_vl && avx512_os_enabled;
- #else
+-#else
++  /* OpenBSD 7.5 doesn't support AVX-512. */
return false;
+-#endif
+ }
  #endif
+ 

Re: -current Haskell ports aborting with SIGILL

[James Cook]

On Fri, Apr 19, 2024 at 12:04:54AM +, James Cook wrote:
> On Sun, Feb 18, 2024 at 08:35:26AM -0800, Evan Silberman wrote:
> > Stuart Henderson  wrote:
> > > On 2024/02/18 09:02, Stuart Henderson wrote:
> > > > On 2024/02/17 22:08, Greg Steuck wrote:
> > > > > Oh wow, this is becoming eerily similar to the failures aja@ is 
> > > > > getting. Do
> > > > > dig more into this!
> > > > 
> > > > Antoine, can you send a dmesg from one of the exopi VMs, please?
> > > 
> > > - specifically I am wondering if it could be someething to do with AVX,
> > > with AVX512 being the most likely to cause problems - Evan's machine
> > > does have this - my intel 11th gen doesn't because it has a mix of
> > > P+E cores, E cores don't implement it, so they disable it on the P
> > > cores too.
> > > 
> > > ghc *does* have some code relating to AVX512.
> > 
> > Breakthrough, ignore the previous reproducer and any association with
> > template haskell. I can get a crash in GHCI very simply:
> > 
> > ~ $ ghci
> > GHCi, version 9.6.4: https://www.haskell.org/ghc/  :? for help
> > ghci> import qualified Data.Text as T
> > ghci> T.take 1 $ T.pack "aa"
> > "Illegal instruction (core dumped)
> 
> I'm seeing this on my OpenBSD 7.5 machine with an old AMD cpu (dmesg
> follows signature). I have ghc-9.6.4p2 (which I think is after the
> fix on this thread). Also pandoc doesn't work.
> 
> $ pkg_info -I ghc pandoc
> ghc-9.6.4p2 compiler for the functional language Haskell
> pandoc-3.1.12.2 convert between markup and document formats
> $ echo | pandoc -o tmp/test.html 
> Illegal instruction (core dumped) 
> $ ghci
> GHCi, version 9.6.4: https://www.haskell.org/ghc/  :? for help
> ghci> import qualified Data.Text as T
> ghci> T.take 1 $ T.pack "aa"
> "Illegal instruction (core dumped) 
> $ 
> 
> I also saw ghc die with a SIGILL when I tried to build pandoc from
> ports (checked out from cvs with -rOPENBSD_7_5). It happened when
> cabal was trying to build unicode-collation-0.1.3.6.

Here are some results of debugging with lldb.


With cabal-bundler and pandoc, it seems to be the xgetbv instruction
itself:


$ lldb /usr/local/bin/cabal-bundler
(lldb) target create "/usr/local/bin/cabal-bundler"
Current executable set to '/usr/local/bin/cabal-bundler' (x86_64).
(lldb) run
Process 90738 launched: '/usr/local/bin/cabal-bundler' (x86_64)
Process 90738 stopped
* thread #1, stop reason = signal SIGILL
frame #0: 0x004c12ba cabal-bundler`___lldb_unnamed_symbol522 + 90
cabal-bundler`___lldb_unnamed_symbol522:
->  0x4c12ba <+90>: xgetbv 
0x4c12bd <+93>: notl   %eax
0x4c12bf <+95>: testb  $-0x20, %al
0x4c12c1 <+97>: leaq   0x58(%rip), %rcx  ; ___lldb_unnamed_symbol523


$ lldb pandoc /dev/null
(lldb) target create "pandoc"
Current executable set to '/usr/local/bin/pandoc' (x86_64).
(lldb) settings set -- target.run-args  "/dev/null"
(lldb) run
Process 25189 launched: '/usr/local/bin/pandoc' (x86_64)
[WARNING] Could not deduce format from file extension 
  Defaulting to markdown
Process 25189 stopped
* thread #1, stop reason = signal SIGILL
frame #0: 0x057697fa pandoc`___lldb_unnamed_symbol1367 + 90
pandoc`___lldb_unnamed_symbol1367:
->  0x57697fa <+90>: xgetbv 
0x57697fd <+93>: notl   %eax
0x57697ff <+95>: testb  $-0x20, %al
0x5769801 <+97>: leaq   0x58(%rip), %rcx  ; 
___lldb_unnamed_symbol1368
(lldb) 


I tried doing the same for Evan's ghci example, but lldb did not
automatically print assembly output as it did for pandoc. I don't
really know how to use lldb. I tried the "di" command but I don't
know if it's doing the right thing:

$ lldb /usr/local/lib/ghc-9.6.4/bin/ghc-9.6.4 -- --interactive 
(lldb) target create "/usr/local/lib/ghc-9.6.4/bin/ghc-9.6.4"
Current executable set to '/usr/local/lib/ghc-9.6.4/bin/ghc-9.6.4' (x86_64).
(lldb) settings set -- target.run-args  "--interactive"
(lldb) run
Process 14800 launched: '/usr/local/lib/ghc-9.6.4/bin/ghc-9.6.4' (x86_64)
GHCi, version 9.6.4: https://www.haskell.org/ghc/  :? for help
ghci> import qualified Data.Text as T
ghci> T.take 1 $ T.pack "aa"
"Process 14800 stopped
* thread #1, stop reason = signal SIGILL
frame #0: 0x0002d2a1242b libc.so.99.0`_thread_sys_futex at -:2
warning: This version of LLDB has no plugin for the language "assembler". 
Inspection of frame variables will be limited.
(lldb) di
libc.so.99.0`_thread_sys_futex:
0x2d2a12410 <+0>:  endbr64 
0x2d2a12414 <+4>:  movq   0x849ad(%rip), %r11

Re: -current Haskell ports aborting with SIGILL

[James Cook]

On Sun, Feb 18, 2024 at 08:35:26AM -0800, Evan Silberman wrote:
> Stuart Henderson  wrote:
> > On 2024/02/18 09:02, Stuart Henderson wrote:
> > > On 2024/02/17 22:08, Greg Steuck wrote:
> > > > Oh wow, this is becoming eerily similar to the failures aja@ is 
> > > > getting. Do
> > > > dig more into this!
> > > 
> > > Antoine, can you send a dmesg from one of the exopi VMs, please?
> > 
> > - specifically I am wondering if it could be someething to do with AVX,
> > with AVX512 being the most likely to cause problems - Evan's machine
> > does have this - my intel 11th gen doesn't because it has a mix of
> > P+E cores, E cores don't implement it, so they disable it on the P
> > cores too.
> > 
> > ghc *does* have some code relating to AVX512.
> 
> Breakthrough, ignore the previous reproducer and any association with
> template haskell. I can get a crash in GHCI very simply:
> 
> ~ $ ghci
> GHCi, version 9.6.4: https://www.haskell.org/ghc/  :? for help
> ghci> import qualified Data.Text as T
> ghci> T.take 1 $ T.pack "aa"
> "Illegal instruction (core dumped)

I'm seeing this on my OpenBSD 7.5 machine with an old AMD cpu (dmesg
follows signature). I have ghc-9.6.4p2 (which I think is after the
fix on this thread). Also pandoc doesn't work.

$ pkg_info -I ghc pandoc
ghc-9.6.4p2 compiler for the functional language Haskell
pandoc-3.1.12.2 convert between markup and document formats
$ echo | pandoc -o tmp/test.html 
Illegal instruction (core dumped) 
$ ghci
GHCi, version 9.6.4: https://www.haskell.org/ghc/  :? for help
ghci> import qualified Data.Text as T
ghci> T.take 1 $ T.pack "aa"
"Illegal instruction (core dumped) 
$ 

I also saw ghc die with a SIGILL when I tried to build pandoc from
ports (checked out from cvs with -rOPENBSD_7_5). It happened when
cabal was trying to build unicode-collation-0.1.3.6.

I haven't followed the thread closely enough to understand what's
going on, but hopefully this is helpful. I'm pretty sure pandoc
worked on this machine when it ran OpenBSD 7.4.

-- 
James


OpenBSD 7.5 (GENERIC.MP) #82: Wed Mar 20 15:48:40 MDT 2024
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 17160474624 (16365MB)
avail mem = 16619225088 (15849MB)
random: good seed from bootblocks
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.4 @ 0xf0100 (59 entries)
bios0: vendor Award Software International, Inc. version "F7" date 11/20/2009
bios0: Gigabyte Technology Co., Ltd. GA-MA790XT-UD4P
acpi0 at bios0: ACPI 1.0
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP SSDT HPET MCFG TAMG APIC
acpi0: wakeup devices PCI0(S5) USB0(S3) USB1(S3) USB2(S3) USB3(S3) USB4(S3) 
USB5(S3) USB6(S3) SBAZ(S4) P2P_(S5) PCE2(S4) PCE3(S4) PCE4(S4) PCE5(S4) 
PCE6(S4) PCE7(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 32 bits
acpihpet0 at acpi0: 14318180 Hz
acpimcfg0 at acpi0
acpimcfg0: addr 0xe000, bus 0-255
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: AMD Phenom(tm) II X3 710 Processor, 2611.95 MHz, 10-04-02, patch 01db
cpu0: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,MWAIT,CX16,POPCNT,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,3DNOW2,3DNOW,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,HWPSTATE,ITSC
cpu0: 64KB 64b/line 2-way D-cache, 64KB 64b/line 2-way I-cache
cpu0: 512KB 64b/line 16-way L2 cache
cpu0: smt 0, core 0, package 0
cpu0: AMD erratum 721 detected and fixed
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 200MHz
cpu0: mwait min=64, max=64, IBE
cpu1 at mainbus0: apid 1 (application processor)
cpu1: AMD Phenom(tm) II X3 710 Processor, 2611.97 MHz, 10-04-02, patch 01db
cpu1: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,MWAIT,CX16,POPCNT,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,3DNOW2,3DNOW,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,HWPSTATE,ITSC
cpu1: 64KB 64b/line 2-way D-cache, 64KB 64b/line 2-way I-cache
cpu1: 512KB 64b/line 16-way L2 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 2 (application processor)
cpu2: AMD Phenom(tm) II X3 710 Processor, 2612.03 MHz, 10-04-02, patch 01db
cpu2: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,MWAIT,CX16,POPCNT,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,3DNOW2,3DNOW,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,HWPSTATE,ITSC
cpu2: 64KB 64b/line 2-way D-cache, 64KB 64b/line 2-way I-cache
cpu2: 512KB 64b/line 16-way L2 cache
cpu2: smt 0, core 2, package 0
ioapic0 at mainbus0: apid 2 pa 0xfec0, version 21, 24 pins, remapped
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 4 (P2P_)
acpiprt2 at acpi0: bus 1 (PCE2)
acpiprt3 at acpi0: bus -1 (PCE3)
acpiprt4 at acpi0: bus 2 (PCE4)
acpiprt5 at acpi0: bus -1 (PCE5)

Re: [new/wip] sysutils/apmtop

[James Cook]

I tried it on my desktop and I can say it seems to mostly work.  It
report the cpu temperature as -1 C which I assume means 'not available'.
There is actually one sensor on this machine that reports a temperature,
but since it doesn't contain the "cpu" string in its name it gets
skipped.

% sysctl -a | grep degC
hw.sensors.ksmn0.temp0=25.25 degC (Tctl)


Oh, I didn't know about hw.sensors.

I see

hw.sensors.cpu0.temp0=38.00 degC
hw.sensors.pchtemp0.temp0=37.00 degC

but apmtop is showing "CPU: 31 C".

--
James

Re: [new/wip] sysutils/apmtop

[James Cook]

Thanks, I tried it out on my laptop.

Thanks for trying.


The BAT and PWR displays always show 0% and 0mW respectively as far 
as I can tell. Note "apm -l" can show my batter percentage so I 
guess that at least should be possible to get right. dmesg below in 
case it's useful.

Now it's fixed.
I forgot to unveil("/dev/apm", "r").
See attached port.


Now BAT works (matches apm -l). Power still shows 0mW 
(min: 0/max: 0) but I don't even know if my laptop can report power 
consumption.



CPU percentage is lower than what "top -1" shows. E.g. with
"++$x while 1" running in a perl repl, top -1 shows 19% user but 
your tool shows about 10%. I don't know which is more accurate, so 
this isn't necessarily a complaint.

Try with a different delay, maybe the same one as top uses.
Use `apmtop -d DELAY` with DELAY being measured in tenths of seconds.
e.g. `apmtop -d 10` refreshes every second.


Comparing "apmtop -d 10" with "top -1s1" it's still a bit different, 
but I don't have any evidence top's estimate is better.


I'm running "find /" in an xterm to use the CPU a bit.

Two side-by-side "top -1s1" instances agree within a couple percentage 
points. "apmtop -d 10" shows a different number, e.g. 21% when top 
shows 32%. Usually lower but sometimes higher.


CPU frequency looks plausible. Temperature is at least changing 
(went from 32 C to 33 C); not sure if it's accurate.

I'll look into that.


I'm not saying 32C sounds wrong, just that I don't have 
a way to verify it :)


--
James

Re: [new/wip] sysutils/apmtop

[James Cook]

On Thu, Jan 04, 2024 at 10:24:18PM +0100, Benjamin Stürz wrote:

Hi ports@,

this is my next little project, after sysutils/lsblk.

apmtop is a simple utility for measuring power consumption
and statistics related to it, in a little TUI interface.
There is still some work to do, but I wanted to see
if anyone would be interested in using/testing this project.

Obviously, this is not (yet) commit-ready, so:
Comments and Suggestions are _very_ welcome!

WWW: https://got.stuerz.xyz/?action=summary=apmtop.git

PS:
The keybindings are in the README.md, until I write a man page.

Happy new year,
Benjamin Stürz


Thanks, I tried it out on my laptop.

The BAT and PWR displays always show 0% and 0mW respectively as far 
as I can tell. Note "apm -l" can show my batter percentage so I 
guess that at least should be possible to get right. dmesg below 
in case it's useful.


CPU percentage is lower than what "top -1" shows. E.g. with
"++$x while 1" running in a perl repl, top -1 shows 19% user but 
your tool shows about 10%. I don't know which is more accurate, so 
this isn't necessarily a complaint.


CPU frequency looks plausible. Temperature is at least changing 
(went from 32 C to 33 C); not sure if it's accurate.


Nice utility! Don't know if I'll use it since I like to squeeze 
such things into my status bar, but maybe I can learn from it. One 
annoyance: xterm flashes on every update as the TUI is redrawn, 
which is distracting when I'm trying to look at something else.


--
James


OpenBSD 7.4-current (GENERIC.MP) #1578: Thu Dec 28 00:15:55 MST 2023
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 16837365760 (16057MB)
avail mem = 16307105792 (15551MB)
random: good seed from bootblocks
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 3.2 @ 0x5e6c6000 (94 entries)
bios0: vendor Dell Inc. version "1.7.0" date 10/22/2020
bios0: Dell Inc. XPS 13 7390
efi0 at bios0: UEFI 2.7
efi0: Dell rev 0x1
acpi0 at bios0: ACPI 6.1
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP SSDT SSDT HPET APIC MCFG SSDT SSDT NHLT SSDT SSDT TPM2 
LPIT SSDT DBGP DBG2 BOOT SSDT DMAR SSDT BGRT FPDT
acpi0: wakeup devices XHC_(S0) XDCI(S4) HDAS(S4) RP01(S4) PXSX(S4) RP02(S4) 
PXSX(S4) RP03(S4) PXSX(S4) RP04(S4) PXSX(S4) RP05(S4) PXSX(S4) RP06(S4) 
PXSX(S4) RP07(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpihpet0 at acpi0: 2399 Hz
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM) i7-10710U CPU @ 1.10GHz, 2893.26 MHz, 06-a6-00, patch 
00f8
cpu0: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SGX,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PT,SRBDS_CTRL,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,IBRS_ALL,SKIP_L1DFL,MDS_NO,MISC_PKG_CT,ENERGY_FILT,FB_CLEAR,RRSBA,GDS_CTRL,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu0: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 256KB 64b/line 
4-way L2 cache, 12MB 64b/line 16-way L3 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 24MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4.1.1.1, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Core(TM) i7-10710U CPU @ 1.10GHz, 2893.26 MHz, 06-a6-00, patch 
00f8
cpu1: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SGX,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PT,SRBDS_CTRL,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,IBRS_ALL,SKIP_L1DFL,MDS_NO,MISC_PKG_CT,ENERGY_FILT,FB_CLEAR,RRSBA,GDS_CTRL,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu1: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 256KB 64b/line 
4-way L2 cache, 12MB 64b/line 16-way L3 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Core(TM) i7-10710U CPU @ 1.10GHz, 2893.26 MHz, 06-a6-00, patch 
00f8
cpu2: 

Re: Upgrade git-annex to 10.20230802

[James Cook]

On Tue, Nov 28, 2023 at 09:05:04PM -0800, Greg Steuck wrote:

Thank you for giving these a try. Care to try again with just the single
patch included?


Oops, yes, was applying both patches before. With just this one, 
it's working fine so far. Will report back if I have trouble.


Thanks!

--
James

Re: Bump darcs for ghc-9.6

[James Cook]

On Sun, Nov 12, 2023 at 10:14:07AM -0800, Greg Steuck wrote:
>Took a bit of effort to patch because the upstream doesn't seem to be
>active. Next time I have to touch darcs I'll propose to delete it.
>
>OK?

Is this meant to build with current ports tree? Running "make" under
devel/darcs" I get an error; end of output on terminal included
below.

I still use darcs. Let me know next time it's making trouble and
maybe I can find some time to fix it.


Hunk #1 succeeded at 45.
done
===>   Ignoring patchfile patch-src_Darcs_Util_Tree_Monad_hs.orig
===>  Compiler link: clang -> /usr/bin/clang
===>  Compiler link: clang++ -> /usr/bin/clang++
===>  Compiler link: cc -> /usr/bin/cc
===>  Compiler link: c++ -> /usr/bin/c++
===>  Generating configure for darcs-2.16.5p2
===>  Configuring for darcs-2.16.5p2
===>  Building for darcs-2.16.5p2
Warning: No remote package servers have been specified. Usually you would have
one specified in the config file.
Resolving dependencies...
Error: cabal: Could not resolve dependencies:
[__0] trying: bifunctors-5.6.1 (user goal)
[__1] unknown package: foldable1-classes-compat (dependency of bifunctors)
[__1] fail (backjumping, conflict set: bifunctors, foldable1-classes-compat)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: bifunctors, foldable1-classes-compat

*** Error 1 in . (/usr/ports/devel/cabal/cabal.port.mk:149 'do-build': @cd 
/usr/ports/pobj/darcs-2.16.5/darcs-2.16.5  && /usr/bin/env -i POR...)
*** Error 2 in . (/usr/ports/infrastructure/mk/bsd.port.mk:3055 
'/usr/ports/pobj/darcs-2.16.5/.build_done': @cd /usr/ports/devel/darcs && PK...)
*** Error 2 in /usr/ports/devel/darcs 
(/usr/ports/infrastructure/mk/bsd.port.mk:2702 'all': @lock=darcs-2.16.5p2;  
export _LOCKS_HELD=" darc...)

-- 
James

Re: vis(e) broken: can't load library 'liblua5.4.so.5.4'

[James Cook]

if lua 5.4 is installed at build time:

checking for liblua >= 5.2 ...
checking for lua... no
checking for lua5.4... no
checking for lua5.3... no
checking for lua5.2... no
checking for lua-5.3... no
checking for lua-5.2... no
checking for lua54... yes

regardless of which lua version is used (the vis port specifically asks
for 5.3), vis' configure script needs patching to use the specified
version of lua rather than searching for the 'best' one.

Index: Makefile
===
RCS file: /cvs/ports/editors/vis/Makefile,v
retrieving revision 1.17
diff -u -p -r1.17 Makefile
--- Makefile26 Sep 2023 17:58:06 -  1.17
+++ Makefile4 Oct 2023 16:49:49 -
@@ -2,7 +2,7 @@ COMMENT =   vi-like editor with sam-style

V = 0.8
DISTNAME =  vis-${V}
-REVISION = 0
+REVISION = 1

TEST_V =0.5
TEST_DISTNAME = vis-test-${TEST_V}
@@ -33,6 +33,7 @@ RUN_DEPENDS = devel/lpeg,${MODLUA_FLAVO
TEST_DEPENDS =  editors/vim

CONFIGURE_STYLE =   simple
+CONFIGURE_ENV =LUA_VER=${MODLUA_FLAVOR}
CONFIGURE_ARGS =--prefix=${PREFIX} \
--mandir=${PREFIX}/man

Index: patches/patch-configure
===
RCS file: /cvs/ports/editors/vis/patches/patch-configure,v
retrieving revision 1.6
diff -u -p -r1.6 patch-configure
--- patches/patch-configure 11 Mar 2022 18:58:39 -  1.6
+++ patches/patch-configure 4 Oct 2023 16:49:49 -
@@ -11,3 +11,12 @@ Index: configure
 tryflag CFLAGS -ffunction-sections
 tryflag CFLAGS -fdata-sections
 tryldflag LDFLAGS_AUTO -Wl,--gc-sections
+@@ -443,7 +442,7 @@ int main(int argc, char *argv[]) {
+ }
+ EOF
+
+-  for liblua in lua lua5.4 lua5.3 lua5.2 lua-5.3 lua-5.2 lua54 lua53 
lua52; do
++  for liblua in $LUA_VER; do
+   printf " checking for %s... " "$liblua"
+
+   if test "$have_pkgconfig" = "yes" ; then



Thanks, works for me.

--
James

vis(e) broken: can't load library 'liblua5.4.so.5.4'

[James Cook]

After a recent pkg_add -Dsnap -u, running "vise" from the vis package 
gives:


$ vise
ld.so: vise: can't load library 'liblua5.4.so.5.4'
Killed 

It looks like the vis package depends on the wrong version (5.3.6) 
of lua. If I manually install lua 5.4.6, vise works, but says 
"WARNING: could not find lpeg module" every time it starts up 
(even after I install the lua-lpeg package).


--
James

Re: attempt to fix textproc/wdiff

[James Cook]

On Sun, May 21, 2023 at 03:42:19PM -0400, James Cook wrote:
> On May 18, 2023 3:16:34 p.m. EDT, Omar Polo  wrote:
> >On 2023/05/18 11:08:54 +0200, NilsOla Nilsson  wrote:
> >> On Wed, May 17, 2023 at 07:03:22PM +0200, Omar Polo wrote:
> >> > 
> >> > still in doubt if it's useful to waste spaces on the mirrors for a
> >> > debug package for this.
> >> 
> >> There is also dwdiff in ports, which is probably
> >> better. From the manpage:
> >> dwdiff is mostly command-line compatible with wdiff
> >
> >I've missed that, thanks!
> >
> >I couldn't make it highlight the difference as wdiff does without "|
> >less" but it's not a big deal, and more importantly it doesn't crash
> >out of the box!
> >
> >At this point I'm wondering whether we should remove wdiff and just
> >point users to dwdiff...
> >
> 
> I have not tried dwdiff, but if it can't highlight differences in less 
> output, I would definitely miss that feature. I find it hard to see where the 
> differences are unless they stand out like that; otherwise it's just a jumble 
> of text.
> -- 
> James

dwdiff seems to have this feature: I just tried it briefly, but it looks
like

got diff | dwdiff -ul | less

is a replacement for

got diff | wdiff -ad

or

got diff | wdiff -dl | less

So my first impression is I won't miss wdiff. Thanks Nils!

-- 
James

Re: attempt to fix textproc/wdiff

[James Cook]

On May 18, 2023 3:16:34 p.m. EDT, Omar Polo  wrote:
>On 2023/05/18 11:08:54 +0200, NilsOla Nilsson  wrote:
>> On Wed, May 17, 2023 at 07:03:22PM +0200, Omar Polo wrote:
>> > 
>> > still in doubt if it's useful to waste spaces on the mirrors for a
>> > debug package for this.
>> 
>> There is also dwdiff in ports, which is probably
>> better. From the manpage:
>> dwdiff is mostly command-line compatible with wdiff
>
>I've missed that, thanks!
>
>I couldn't make it highlight the difference as wdiff does without "|
>less" but it's not a big deal, and more importantly it doesn't crash
>out of the box!
>
>At this point I'm wondering whether we should remove wdiff and just
>point users to dwdiff...
>

I have not tried dwdiff, but if it can't highlight differences in less output, 
I would definitely miss that feature. I find it hard to see where the 
differences are unless they stand out like that; otherwise it's just a jumble 
of text.
-- 
James

Re: attempt to fix textproc/wdiff

[James Cook]

On Wed, May 17, 2023 at 07:03:22PM +0200, Omar Polo wrote:
> I discovered wdiff the other day thanks to falsifian, but it has a bad
> habit of crashing repeatedly.
> 
> The TL;DR (and you're really better off not reading the sources) is
> that it malloc(0) and then passes the pointer to tgetstr() where it
> crashes due to a write attempt.
> 
> Here's a way to address it.  To try, both before and after the fix, try
> 
>   $ {got,git,cvs,rcsdiff,whatever} diff | wdiff -ad
> 
> still in doubt if it's useful to waste spaces on the mirrors for a
> debug package for this.

Thanks! This fixes the segfault as far as I can tell.

I noticed something else while trying it, though: without help2man
installed, making the port fails and I see:

/usr/ports/pobj/wdiff-1.2.2/wdiff-1.2.2/build-aux/missing[81]: 
help2man: not found
WARNING: 'help2man' is missing on your system.
 You should only need it if you modified a dependency of a man 
page.
 You may want to install the GNU Help2man package:
 
gmake[2]: *** [Makefile:1238: wdiff.1] Error 127
gmake[2]: Leaving directory 
'/usr/ports/pobj/wdiff-1.2.2/wdiff-1.2.2/man'
gmake[1]: *** [Makefile:1144: all-recursive] Error 1
gmake[1]: Leaving directory '/usr/ports/pobj/wdiff-1.2.2/wdiff-1.2.2'
gmake: *** [Makefile:1084: all] Error 2
*** Error 2 in . (/usr/ports/infrastructure/mk/bsd.port.mk:2961 
'/usr/ports/pobj/wdiff-1.2.2/.build_done': @cd /usr/ports/pobj/wdiff-1.2.2/w...)
*** Error 2 in /usr/ports/textproc/wdiff 
(/usr/ports/infrastructure/mk/bsd.port.mk:2601 'all': @lock=wdiff-1.2.2p4;  
export _LOCKS_HELD=" wd...)

-- 
James

Re: update math/py-scipy: 1.5.3 -> 1.6.3

[James Cook]

Daniel Dickman  writes:
> It's just a warning, right? It works for me here.
>
> If something isn't working on your end can you share a code snippet?

Ah, you're right, it's just a warning.

It does seem to work; I tested using beancount-import. Below is one
unfortunate caveat. I'm not sure if this counts as ports' problem; maybe
you could say it's my fault for mixing pkg_add and pip:

If I do the following:

- (as root) pkg_add some python deps including py3-scikit-learn (which
  depends on scipy)
- (as non-root) pip install beancount-import
- (as non-root, in Python) beancount_import.webserver.main(...)

I see:

RuntimeError: module compiled against API version 0x10 but this version of 
numpy is 0xf . Check the section C-API incompatibility at the Troubleshooting 
ImportError section at 
https://numpy.org/devdocs/user/troubleshooting-importerror.html#c-api-incompatibility
 for indications on how to solve this problem .

and a bunch more error lines cumulating in
ImportError: numpy.core.multiarray failed to import
and the beancount-import webserver drops to a Pdb prompt.

My guess: pip installed numpy-1.22.4 to my home dir because of the
version incompatibility, but did not override the system scipy, leading
to some sort of incompatibility. Evidence for my guess: if I delete
~/.local/python3.10/site-packages/numpy*, the error goes away and
beancount-import works.

-- 
James

Re: update math/py-scipy: 1.5.3 -> 1.6.3

[James Cook]

Daniel Dickman  writes:
> Diff below updates scipy to the 1.6.x series.
>
> With more work, it's possible to update to newer versions of scipy, but 
> for now I went for a more conservative update that doesn't change as much 
> as some of the newer versions change (e.g. 1.7 starts to use pythran, 1.8 
> makes gcc 6.3 the minimum compiler, 1.9 switches the build to meson, 
> etc..)
>
> Tested on amd64 and all reverse deps were built.

scipy no longer works for me, though it looks like the blame is on
numpy:

moth$ python3
Python 3.10.9 (main, Feb  2 2023, 05:09:30) [Clang 13.0.0 ] on openbsd7
Type "help", "copyright", "credits" or "license" for more information.
>>> import scipy
/usr/local/lib/python3.10/site-packages/scipy/__init__.py:138: UserWarning: A 
NumPy version >=1.16.5 and <1.23.0 is required for this version of SciPy 
(detected version 1.24.1)
  warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion} is 
required for this version of "

This is on amd64 -current; I just ran pkg_add -u scipy a few minutes ago
in case the problem didn't get fixed since the last time I tried.

-- 
James

Re: Upgrade git-annex 10.20220822 -> 10.20221103

[James Cook]

Greg Steuck  writes:
> I tested locally with `git-annex test`. Other testing is welcome.
>
> Thanks
> Greg

Thanks! I tried a few basic operations with my git-annex repos and it
seems to work. Will report back if anything goes wrong.

-- 
James

Re: [new] mail/meli: a new terminal email client

[James Cook]

> Thanks for the test report and the extensive testing, but ... you do
> realize all those things you're reporting arent probably related to the
> OpenBSD port itself, and would be better reported upstream at
> https://git.meli.delivery/meli/meli/issues ? After all meli is not even
> published as 'beta', only alpha.. :)
>
> Landry

Reported at:

https://git.meli.delivery/meli/meli/issues/164
https://git.meli.delivery/meli/meli/issues/165

-- 
James

Re: [new] mail/meli: a new terminal email client

[James Cook]

On Wed Oct 26, 2022 at 5:58 AM UTC, Theo Buehler wrote:
> > Thanks for porting. I just tried to use meli with notmuch, and get:
> > 
> > $ meli
> > Could not load libnotmuch!
> > If notmuch is installed but the library isn't found, consult your 
> > system's documentation on how to make dynamic libraries discoverable.
> > Caused by: File not found
> > 
> > I do have notmuch installed, and libnotmuch is at
> > /usr/local/lib/libnotmuch.so.1.2 . I installed meli with pkg_add
> > (meli-0.7.2pl20221017).
> > 
> > I can try to debug but don't currently know much about rust and dynamic
> > libraries.
>
> Does this work?

Thanks, it seems to work. Some things are broken, but some of those
things aren't notmuch-specific and seem to be broken in the package
(installed with pkg_add) too.

Broken things I noticed, for the record:

- In threaded view, I can't seem to change notmuch tags, including
  "unread". "tag add test" doesn't do anything (as measured later by
  running "notmuch search tag:test"). Pressing "n" or opening a message
  doesn't remove the notmuch "unread" tag (and visually it remains
  unread).

- Similarly, in threaded view, pressing "v" to select a message doesn't
  seem to do anything.

- To see if it's notmuch related, I also tried the IMAP backend. With
  IMAP and with listing.index_style = "threadad", meli exits with exit
  code 101 shortly after (successfully) displaying the inbox. With
  the default index_style, I don't see this problem. (I think I saw
  that exit code once when playing with the notmuch backend, but wasn't
  able to reproduce it.) This also happens with the official package
  installed using pkg_add meli.

- The colours are messed up if I try to run it in tmux. Depending on
  theme / tmux fg and bg colours, sometimes the selected message in the
  index is invisible (black text on black bg) and sometimes the
  currently highlighted message in the index doesn't appear any
  different from when it wasn't highlighted.

dmesg below, just in case.

-- 
James


OpenBSD 7.2-current (GENERIC.MP) #776: Sat Oct  8 16:41:49 MDT 2022
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 16837365760 (16057MB)
avail mem = 16309669888 (15554MB)
random: good seed from bootblocks
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 3.2 @ 0x5e6c6000 (94 entries)
bios0: vendor Dell Inc. version "1.7.0" date 10/22/2020
bios0: Dell Inc. XPS 13 7390
acpi0 at bios0: ACPI 6.1
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP SSDT SSDT HPET APIC MCFG SSDT SSDT NHLT SSDT SSDT TPM2 
LPIT SSDT DBGP DBG2 BOOT SSDT DMAR SSDT BGRT FPDT
acpi0: wakeup devices XHC_(S0) XDCI(S4) HDAS(S4) RP01(S4) PXSX(S4) RP02(S4) 
PXSX(S4) RP03(S4) PXSX(S4) RP04(S4) PXSX(S4) RP05(S4) PXSX(S4) RP06(S4) 
PXSX(S4) RP07(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpihpet0 at acpi0: 2399 Hz
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM) i7-10710U CPU @ 1.10GHz, 3890.94 MHz, 06-a6-00
cpu0: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SGX,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PT,SRBDS_CTRL,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu0: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 256KB 64b/line 
4-way L2 cache, 12MB 64b/line 16-way L3 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 24MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4.1.1.1, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Core(TM) i7-10710U CPU @ 1.10GHz, 3890.94 MHz, 06-a6-00
cpu1: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SGX,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PT,SRBDS_CTRL,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu1: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 256KB 64b/line 
4-way L2 cache, 12MB 64b/line 16-way L3 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Core(TM) i7-10710U CPU @ 1.10GHz, 3890.94 MHz, 06-a6-00
cpu2: 

Re: [new] mail/meli: a new terminal email client

[James Cook]

On Tue Oct 18, 2022 at 11:10 AM UTC, Landry Breuil wrote:
> Le Fri, Oct 07, 2022 at 09:50:38AM +0200, Landry Breuil a écrit :
> > Le Thu, Oct 06, 2022 at 02:40:28PM +0200, Landry Breuil a écrit :
> > > Hi,
> > > 
> > > here's a wip port of https://meli.delivery, i've chosen to port the git
> > > head instead of the last-year-old alpha-0.7.2 tag - there's wip support
> > > for jmap (which im also poking at).
> > > 
> > > totally untested for now, but looks nice and simple. It starts and
> > > there's a step-by-step assistant for configuration.
> > > 
> > > https://meli.delivery/documentation.html#quick-start
> > > https://meli.delivery/documentation.html#meli.7 is the full tutorial
> > > 
> > > meli man feature broken for now, since i think it relies on gzipped
> > > manpages, but i'll eventually have a look. plain manpages shipped in the
> > > package works.
> > 
> > meli man failed for me because i had PAGER=less -X in the env, with
> > PAGER=less it works.
> > 
> > new version with feedback from sthen@ and tb@.
> > 
> > runtime testing/feedback welcome, oks too :)
>
> and new version with https://git.meli.delivery/meli/meli/issues/159
> fixed thanks to semarie@ and upstream.
>
> runtime testing shows it works fine, just need to get used to shortcuts.
>
> oks to import welcome :)
>
> Landry

Thanks for porting. I just tried to use meli with notmuch, and get:

$ meli
Could not load libnotmuch!
If notmuch is installed but the library isn't found, consult your 
system's documentation on how to make dynamic libraries discoverable.
Caused by: File not found

I do have notmuch installed, and libnotmuch is at
/usr/local/lib/libnotmuch.so.1.2 . I installed meli with pkg_add
(meli-0.7.2pl20221017).

I can try to debug but don't currently know much about rust and dynamic
libraries.

Here's my meli/config.toml:

[accounts.notmuch]
root_mailbox = "/home/falsifian/var/mbsync/falsifian.org"
format = "notmuch"
listing.index_style = "threaded"
identity="falsif...@falsifian.org"
display_name = "James Cook"
  [accounts.notmuch.mailboxes]
  "INBOX" = {  query="tag:inbox", subscribe = true }
  "Drafts" = {  query="tag:draft", subscribe = true }
  "Reading" = {  query="tag:reading", subscribe = true }
  "Sent" = {  query="folder:Sent", subscribe = true }
  "Spam" = {  query="tag:spam", subscribe = true }

[composing]
send_mail = "sendmail"

-- 
James

Re: bitlbee-facebook: fix URL in DESCR

[James Cook]

> > you need to bump the REVISION of the package to trigger a rebuild
> > with this kind of change. If REVISION doesn't exist, you can start
> > with REVISION=0
>
> yup.  for future refence, when changing something that alters the
> package content (description and comment counts too for example)
> without changing the version number you need to "bump" the REVISION.

Thanks, Solène and Omar. I have seen the same mistake on ports@ enough
times that I should have known.

-- 
James

bitlbee-facebook: fix URL in DESCR

[James Cook]

The patch below fixes a URL. I haven't tested building the port with the
change, but since it's just in DESCR I assume it's fine.

-- 
James

diff --git a/net/bitlbee-facebook/pkg/DESCR b/net/bitlbee-facebook/pkg/DESCR
index 6c11e2f964c..70daa021d6e 100644
--- a/net/bitlbee-facebook/pkg/DESCR
+++ b/net/bitlbee-facebook/pkg/DESCR
@@ -2,4 +2,4 @@ The Facebook protocol plugin for bitlbee. This plugin uses the 
Facebook
 Mobile API.
 
 General usage instructions are available in the bitlbee wiki:
-https://wiki.bitlbee.org/HowtoFacebookMQT
+https://wiki.bitlbee.org/HowtoFacebookMQTT

Re: [new] mail/aerc - terminal email client

[James Cook]

On Mon May 2, 2022 at 12:25 AM UTC, James Cook wrote:
> > It's based on a previous submission by Raymond (+cc).  I've enabled the
> > notmuch support (but not really tried it yet) and updated it, plus some
> > other minor tweaks.
>
> It is working for me with notmuch. I'm sending this with your aerc port.
> This is my first time using aerc. Thanks for porting it!
>
> -- 
> James

There is a quirk which I forgot to mention: when I do a notmuch search
using :cf, the new "folder" is added to the left with the number of
results, but the right pane says "(no messages)". If I :cf with the same
search a second time, the messages appear.

It seems the messages appear as long as I've :cf'd with that specific
search before in the same session; it still works if I do intermediate
searches. Maybe some kind of cacheing is involved?

-- 
James

Re: [new] mail/aerc - terminal email client

[James Cook]

> It's based on a previous submission by Raymond (+cc).  I've enabled the
> notmuch support (but not really tried it yet) and updated it, plus some
> other minor tweaks.

It is working for me with notmuch. I'm sending this with your aerc port.
This is my first time using aerc. Thanks for porting it!

-- 
James

Re: lang/racket-minimal: update to v8.3

[James Cook]

> > but with your user. Running "racket-user-bin-paths" you should see the
> > bin directory path within $HOME/.racket.
> > 
> > And this is the right config file in /etc/racket/config.rktd:
> > ;; generated by unixstyle-install
> > #hash(
> >   (doc-dir . "/usr/local/share/doc/racket")
> >   (lib-dir . "/usr/local/lib/racket")
> >   (pkgs-dir . "/usr/local/share/racket/pkgs")
> >   (share-dir . "/usr/local/share/racket")
> >   (include-dir . "/usr/local/include/racket")
> >   (bin-dir . "/usr/local/bin")
> >   (apps-dir . "/usr/local/share/applications")
> >   (man-dir . "/usr/local/man")
> >   (absolute-installation? . #t)
> >   (compiled-file-roots . (same "/usr/local/lib/racket/compiled"))
> >   (build-stamp . "")
> >   (doc-search-url . 
> > "https://download.racket-lang.org/releases/8.3/doc/local-redirect/index.html;)
> >   (catalogs . ("https://download.racket-lang.org/releases/8.3/catalog/; 
> > #f))
> > )

Oops, I forgot the last two things you asked for:



rt# su falsifian -c racket-user-bin-paths
/home/falsifian/.local/share/racket/8.3/bin:/home/falsifian/.local/share/racket/
8.3/binrt# cat /etc/racket/config.rktd
;; generated by unixstyle-install
#hash(
  (doc-dir . "/usr/local/share/doc/racket")
  (lib-dir . "/usr/local/lib/racket")
  (pkgs-dir . "/usr/local/share/racket/pkgs")
  (share-dir . "/usr/local/share/racket")
  (include-dir . "/usr/local/include/racket")
  (bin-dir . "/usr/local/bin")
  (apps-dir . "/usr/local/share/applications")
  (man-dir . "/usr/local/man")
  (absolute-installation? . #t)
  (compiled-file-roots . (same "/usr/local/lib/racket/compiled"))
  (build-stamp . "")
  (doc-search-url . "https://download.racket-lang.org/releases/8.3/doc/local
-redirect/index.html")
  (catalogs . ("https://download.racket-lang.org/releases/8.3/catalog/; #f))
)


-- 
James

Re: lang/racket-minimal: update to v8.3

[James Cook]

> I don't see anything wrong with that but I don't know why you have that
> dir. Run the command "raco pkg config" as regular user. You should have
> something like this output:
> 
> current-amd64$ raco pkg config
> name:
> 8.3
> catalogs:
> https://download.racket-lang.org/releases/8.3/catalog/
> https://pkgs.racket-lang.org
> https://planet-compats.racket-lang.org
> default-scope:
> user
> download-cache-dir:
> /home/juanfra/.racket/download-cache
> download-cache-max-files:
> 1024
> download-cache-max-bytes:
> 134217728
> git-checkout-credentials:
> trash-max-packages:
> 512
> trash-max-seconds:
> 172800
> network-retries:
> 5
> 
> default-scope should say "user" and the download dir should be the same
> but with your user. Running "racket-user-bin-paths" you should see the
> bin directory path within $HOME/.racket.
> 
> And this is the right config file in /etc/racket/config.rktd:
> ;; generated by unixstyle-install
> #hash(
>   (doc-dir . "/usr/local/share/doc/racket")
>   (lib-dir . "/usr/local/lib/racket")
>   (pkgs-dir . "/usr/local/share/racket/pkgs")
>   (share-dir . "/usr/local/share/racket")
>   (include-dir . "/usr/local/include/racket")
>   (bin-dir . "/usr/local/bin")
>   (apps-dir . "/usr/local/share/applications")
>   (man-dir . "/usr/local/man")
>   (absolute-installation? . #t)
>   (compiled-file-roots . (same "/usr/local/lib/racket/compiled"))
>   (build-stamp . "")
>   (doc-search-url . 
> "https://download.racket-lang.org/releases/8.3/doc/local-redirect/index.html;)
>   (catalogs . ("https://download.racket-lang.org/releases/8.3/catalog/; 
> #f))
> )
> 
> -- 
> Juan Francisco Cantero Hurtado http://juanfra.info

For reproducibility, I wrote the below script and created another new
VM.


#!/bin/sh
sysctl kern.version
pkg_add racket-minimal-8.3
su falsifian -c "raco pkg install base"
su falsifian -c "raco pkg config"


I ran the script as root on another freshly created OpenBSD vm. The
script was the first command run on the system, except for scp and
chmod to set up the script.

After running the script I confirmed that
/home/falsifian/.local/share/racket/8.3 exists.

Here is the full output. I noticed that download-cache-dir is different
from yours.


kern.version=OpenBSD 7.0-current (GENERIC.MP) #168: Tue Dec 14 15:32:01 MST 2021
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP

quirks-4.89 signed on 2021-12-15T15:12:59Z
New and changed readme(s):
/usr/local/share/doc/pkg-readmes/racket-minimal
Resolving "base" via https://download.racket-lang.org/releases/8.3/catalog/
Downloading https://download.racket-lang.org/releases/8.3/pkgs/base.zip
raco setup: version: 8.3
raco setup: platform: x86_64-openbsd [cs]
raco setup: target machine: ta6ob
raco setup: installation name: 8.3
raco setup: variants: cs
raco setup: main collects: /usr/local/share/racket/collects/
raco setup: collects paths: 
raco setup:   /home/falsifian/.local/share/racket/8.3/collects
raco setup:   /usr/local/share/racket/collects/
raco setup: main pkgs: /usr/local/share/racket/pkgs
raco setup: pkgs paths: 
raco setup:   /usr/local/share/racket/pkgs
raco setup:   /home/falsifian/.local/share/racket/8.3/pkgs
raco setup: links files: 
raco setup:   /usr/local/share/racket/links.rktd
raco setup:   /home/falsifian/.local/share/racket/8.3/links.rktd
raco setup: compiled-file roots: 
raco setup:   same
raco setup:   /usr/local/lib/racket/compiled
raco setup: main docs: /usr/local/share/doc/racket
raco setup: --- updating info-domain tables ---[0:53:11]
raco setup: --- pre-installing collections --- [0:53:11]
raco setup: --- installing foreign libraries ---   [0:53:11]
raco setup: --- installing shared files ---[0:53:11]
raco setup: --- compiling collections ---  [0:53:11]
raco setup: --- parallel build using 4 jobs ---[0:53:11]
raco setup: --- creating launchers --- [0:53:13]
raco setup: --- installing man pages ---   [0:53:13]
raco setup: --- installing collections --- [0:53:13]
raco setup: --- post-installing collections ---[0:53:13]
name:
  8.3
catalogs:
  https://download.racket-lang.org/releases/8.3/catalog/
  https://pkgs.racket-lang.org
  https://planet-compats.racket-lang.org
default-scope:
  user
download-cache-dir:
  /home/falsifian/.cache/racket/download-cache
download-cache-max-files:
  1024
download-cache-max-bytes:
  134217728
git-checkout-credentials:
trash-max-packages:
  512
trash-max-seconds:
  172800
network-retries:
  5

-- 
James

Re: lang/racket-minimal: update to v8.3

[James Cook]

On Tue, Dec 14, 2021 at 09:44:03PM +0100, Juan Francisco Cantero Hurtado wrote:
> On Tue, 14 Dec 2021 14:18:39 +
> James Cook  wrote:
> 
> > On Tue, Dec 14, 2021 at 10:11:12AM +0100, Denis Fondras wrote:
> > > Le Tue, Dec 14, 2021 at 05:27:26AM +, James Cook a écrit :  
> > > > On Sun, Nov 28, 2021 at 10:46:28AM +0100, Denis Fondras wrote:  
> > > > > Straightforward update to v8.3  
> > > > 
> > > > It looks like the update to 8.3 broke the "raco exe" command.
> > > >   
> > > 
> > > I am surprised it ever worked. Minimal installation does not seem
> > > to provide it.  
> > 
> > Oops, sorry for the noise. After "raco pkg install compiler" I have
> > the raco exe command. And conversely, with 8.2, if I delete
> > ~/.local/share/racket, the "raco exe" command also stops working, so
> > this is nothing new.
> > 
> 
> You have something wrong in your config (or you changed the defaults).
> Our racket-minimal doesn't install files in ~/.local/share/racket.

I just did a little experiment. On a brand new vm running current,
"raco pkg install base" causes ~/.local/share/racket to be created. Is
that unexpected?

My steps:

1. Installed the latest snapshot on a new virtual machine.
2. As root, pkg_add racket-minimal. (At the prompt, I chose
   "racket-minimal-8.3" rather than the no-jit flavour.)
3. As non-root, "raco pkg install base".

Result: in the non-root user's home dir, there's now
~/.local/share/racket/8.3

Based on that experiment, it does not seem strange to me that my
existing system had ~/.local/share/racket. And it has not caused me any
trouble. Am I doing something wrong?

-- 
James

Re: lang/racket-minimal: update to v8.3

[James Cook]

On Tue, Dec 14, 2021 at 10:11:12AM +0100, Denis Fondras wrote:
> Le Tue, Dec 14, 2021 at 05:27:26AM +0000, James Cook a écrit :
> > On Sun, Nov 28, 2021 at 10:46:28AM +0100, Denis Fondras wrote:
> > > Straightforward update to v8.3
> > 
> > It looks like the update to 8.3 broke the "raco exe" command.
> > 
> 
> I am surprised it ever worked. Minimal installation does not seem to provide 
> it.

Oops, sorry for the noise. After "raco pkg install compiler" I have the
raco exe command. And conversely, with 8.2, if I delete
~/.local/share/racket, the "raco exe" command also stops working, so
this is nothing new.

-- 
James

Re: lang/racket-minimal: update to v8.3

[James Cook]

On Sun, Nov 28, 2021 at 10:46:28AM +0100, Denis Fondras wrote:
> Straightforward update to v8.3

It looks like the update to 8.3 broke the "raco exe" command.

angel ~ $ raco exe
/usr/local/bin/raco: Unrecognized command: exe

-- 
James

Re: neomutt segmentation fault

[James Cook]

> > falsifian moth ~ $ pkg_info neomutt
> > Information for inst:neomutt-20211015
> 
> So it seems you're running -current, right?

Yes.

> > falsifian moth ~ $ dmesg|head -n1
> > OpenBSD 7.0 (GENERIC.MP) #232: Thu Sep 30 14:25:29 MDT 2021
> > 
> > falsifian moth ~ $ uname -a
> > OpenBSD moth.falsifian.org 7.0 GENERIC.MP#43 amd64
> 
> These two kernel lines don't match and don't immediately tell me if
> you're on release or -current. 'sysctl kern.version' is better.

Sorry, the dmesg line must have been preserved from a previous boot.

falsifian moth ~ $ sysctl kern.version
kern.version=OpenBSD 7.0-current (GENERIC.MP) #43: Mon Oct 18 07:58:26 MDT 2021
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP

> The diff below should fix your problem. Could you check, please?

Yes, that fixes it, thanks. Using it now.

-- 
James

neomutt segmentation fault

[James Cook]

I don't know if this is related to the other recent neomutt segfault
thread.

After sysupgrade -s and pkg_add -u, neomutt crashes immediately when I
start it. Is anyone else seeing this? Details below. I took filesystem
backups before upgrading, so I can probably figure out the exact
(working) neomutt version I had before if that's useful.

I guess my next step will be to see whether it's related to my
some particular part of my .neomuttrc.

falsifian moth ~ $ pkg_info neomutt
Information for inst:neomutt-20211015

...

falsifian moth ~ $ egdb neomutt
GNU gdb (GDB) 7.12.1
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-openbsd7.0".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
.
Find the GDB manual and other documentation resources online at:
.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from neomutt...(no debugging symbols found)...done.
(gdb) run
Starting program: /usr/local/bin/neomutt
[New process 13376]

Thread 1 received signal SIGSEGV, Segmentation fault.
_libc_freeaddrinfo (ai=0x0) at /usr/src/lib/libc/net/freeaddrinfo.c:46
46  free(p->ai_canonname);
(gdb) bt
#0  _libc_freeaddrinfo (ai=0x0) at /usr/src/lib/libc/net/freeaddrinfo.c:46
#1  0x079ac809e2b6 in getdnsdomainname ()
#2  0x079ac800ee1f in mutt_init ()
#3  0x079ac80171f4 in main ()
(gdb)

falsifian moth ~ $ dmesg|head -n1
OpenBSD 7.0 (GENERIC.MP) #232: Thu Sep 30 14:25:29 MDT 2021

falsifian moth ~ $ uname -a
OpenBSD moth.falsifian.org 7.0 GENERIC.MP#43 amd64

Let me know if more details would be useful.

-- 
James

Re: [PATCH 7/9] Upgrade devel/git-annex 8.20210223->8.20211011

[James Cook]

On Sun, Oct 17, 2021 at 08:39:12PM -0700, Greg Steuck wrote:
> I only built this and confirmed the executable is doesn't die right
> away. Somebody who cares should test.

I did a few basic operations and it seems to work so far. I'll let you
know if something goes wrong.

-- 
James

Re: Why do the gtk+ ports use emacs keybindings?

[James Cook]

On Sun, Jul 18, 2021 at 04:58:09PM +0200, Christian Weisgerber wrote:
> Kurt Miller:
> 
> > > x11/gtk+2 and x11/gtk+3 include a patch to change the default
> > > keybindings to "emacs". The decision can be traced back to 2005:
> > > 
> > >Revision 1.1, Wed Sep 7 20:51:14 2005 UTC (15 years, 10 months ago) by 
> > > kurt
> > >Branch: MAIN
> > > 
> > >make emacs the default key theme. this will make the standard ^U, ^A, 
> > > ^E
> > >etc emacs commands work by default for gtk+2 apps. orig from pvalchev@.
> > 
> > I have absolutely no recollection of this commit. I defer to the other
> > port maintainers on this issue.
> 
> I dimly remember... well, not really.
> 
> *checks old ICB logs*
> 
> So, when Firefox switched to GTK+2, various developers were annoyed
> that the Emacs-style keybindings were no longer available, and this
> default was added.
> 
> -- 
> Christian "naddy" Weisgerber  na...@mips.inka.de

Thanks for the back story, Christian.

I guess the general sentiment on this thread is the port should be kept
as-is.

I just remembered another thing, which I'll share in case it sways any
opinions:

When I ran into this, it wasn't "Weird, why is Firefox using emacs key
bindings?" It was: "Why can't I press ctrl-F to search? Why is my
cursor stuck in the address bar, even after I press escape? Why do all
the keyboard shortcuts stop working when that happens?" It was a
collection of bizarre symptoms, making me think the Firefox port was
buggy, and "emacs" never occurred to me until I did some careful web
searching.

In short, it is hard for the new user to debug. And I'm an emacs user.

-- 
James

Why do the gtk+ ports use emacs keybindings?

[James Cook]

Sorry if this has been discussed before; I couldn't find it.

x11/gtk+2 and x11/gtk+3 include a patch to change the default
keybindings to "emacs". The decision can be traced back to 2005:

Revision 1.1, Wed Sep 7 20:51:14 2005 UTC (15 years, 10 months ago) by kurt
Branch: MAIN

make emacs the default key theme. this will make the standard ^U, ^A, ^E
etc emacs commands work by default for gtk+2 apps. orig from pvalchev@.

okay marcm@

16 years later, I wonder if it's worth revisiting this decision.

For my part, I'm strongly opposed to it, but I'm just some user who
occasionally contributes. Still, if you'll hear them, here are my
arguments:

1. This seems to create some confusion, at least for firefox users, e.g:

   
https://www.reddit.com/r/openbsd/comments/4aj3rb/how_do_i_turn_off_emacs_line_editing_in/

   https://mas.to/@adnan360/106589261557989696

   It certainly confused me, until I found the solution through some
   persistent Google searching.

2. Even if I were clever enough to trace the Firefox issue to gtk+3,
   the gtk+3 package README does not describe the problem.

   I'm happy to submit a patch to the gtk+3 README, but in my opinion
   this will not solve the confusion, because it's far from obvious
   that one should check the gtk+3 REDAME if one's Firefox keybindings
   are behaving strangely.

3. I generally don't expect ports to adjust behaviour without a good
   reason. Users have strong expectations about the behaviour of gtk+
   applications (well, Firefox) based on what they've seen on other
   platforms. If you want emacs keybindings, you can edit your own
   config file.

-- 
James

Re: Problem getting unsigned package working on new port

[James Cook]

On Thu, May 06, 2021 at 08:58:40PM -0500, Chris Bennett wrote:
> On Fri, May 07, 2021 at 12:58:18AM +0000, James Cook wrote:
> > On Thu, May 06, 2021 at 07:02:10PM -0500, Chris Bennett wrote:
> > > I'm having an error getting doas make install. Also fails when using root
> > > for make package and when I moved the directory over into 
> > > /usr/ports/devel/
> > > 
> > > This is what is getting produced with make package:
> > > 
> > > CX p5-File-PathList $ tar zvf 
> > > /usr/packages/amd64/all/p5-File-PathList-1.04.tgz
> > > -r--r--r--  1 root wheel  518 May  6 17:47 +CONTENTS
> > > -r--r--r--  1 root wheel  602 May  6 17:47 +DESC
> > > -r--r--r--  1 root bin   5191 Dec 31  1969 
> > > libdata/perl5/site_perl/File/PathList.pm
> > > -r--r--r--  1 root bin   6216 Dec 31  1969 
> > > man/man3p/File::PathList.3p
> > > 
> > > CX all # tar xzvf p5-File-PathList-1.04.tgz
> > > +CONTENTS
> > > +DESC
> > > libdata/perl5/site_perl/File/PathList.pm
> > > man/man3p/File::PathList.3p
> > > 
> > > CX all # cat +CONTENTS
> > > @comment $OpenBSD: PLIST,v$
> > > @name p5-File-PathList-1.04
> > > @comment pkgpath=devel/p5-File-PathList ftp=yes
> > > @arch *
> > > +DESC
> > > @sha zVmYJtE6NhOnNIiaIO/Mg3dqWHUzbGSvEs4uCd9lAvQ=
> > > @size 602
> > > @depend devel/p5-Params-Util:p5-Params-Util-*:p5-Params-Util-1.07p2
> > > @cwd /usr/local
> > > libdata/perl5/site_perl/File/
> > > libdata/perl5/site_perl/File/PathList.pm
> > > @sha xPu9e1UGezN3CF1fdqjA9eWCPK0yALnGUj/3DA/UFjI=
> > > @size 5191
> > > @ts 1215675005
> > > @man man/man3p/File::PathList.3p
> > > @sha v3i5tb/ui1rMuD1cs5tt3H2RWq1mDmy1WGdc5B9cCUM=
> > > @size 6216
> > > @ts 1620344553
> > > 
> > > 
> > > I'm getting the following when doing doas make install:
> > > 
> > > CX p5-File-PathList $ MI
> > > doas (s...@mail.bennettconstruction.us) password: 
> > > ===>  Installing p5-File-PathList-1.04 from /usr/packages/amd64/all/
> > > doas (r...@mail.bennettconstruction.us) password: 
> > > quirks-4.9 signed on 2021-05-06T19:40:58Z
> > > quirks-4.6->4.9: ok
> > > file:/usr/packages/amd64/all/p5-File-PathList-1.04.tgz: unsigned package
> > 
> > I had a similar error, and fixed it by adding keepenv to my doas
> > config. I think the variable that needs to be passed through is
> > TRUSTED_PKG_PATH.
> > 
> > http://openbsd-archive.7691.n7.nabble.com/signify-error-when-installing-ports-on-current-td366895.html
> > 
> 
> I also thought that TRUSTED_PKG_PATH was likely the problem.
> 
> What exactly did you put in doas.conf?

I just have:

  permit persist keepenv falsifian

  # Adapted from the bsd.port.mk man page.
  permit keepenv nopass falsifian as _pbuild
  permit keepenv nopass falsifian as _pfetch

> I also untarred -current's /etc/example/doas.conf and didn't see anything 
> different from
> what I already had uncommented.
> 
> When I run make show=TRUSTED_PKG_PATH under /usr/ports/devel I get it
> from every port except my new one.

Sorry, this is beyond me.

-- 
James

Re: Problem getting unsigned package working on new port

[James Cook]

On Thu, May 06, 2021 at 07:02:10PM -0500, Chris Bennett wrote:
> I'm having an error getting doas make install. Also fails when using root
> for make package and when I moved the directory over into /usr/ports/devel/
> 
> This is what is getting produced with make package:
> 
> CX p5-File-PathList $ tar zvf 
> /usr/packages/amd64/all/p5-File-PathList-1.04.tgz
> -r--r--r--  1 root wheel  518 May  6 17:47 +CONTENTS
> -r--r--r--  1 root wheel  602 May  6 17:47 +DESC
> -r--r--r--  1 root bin   5191 Dec 31  1969 
> libdata/perl5/site_perl/File/PathList.pm
> -r--r--r--  1 root bin   6216 Dec 31  1969 
> man/man3p/File::PathList.3p
> 
> CX all # tar xzvf p5-File-PathList-1.04.tgz
> +CONTENTS
> +DESC
> libdata/perl5/site_perl/File/PathList.pm
> man/man3p/File::PathList.3p
> 
> CX all # cat +CONTENTS
> @comment $OpenBSD: PLIST,v$
> @name p5-File-PathList-1.04
> @comment pkgpath=devel/p5-File-PathList ftp=yes
> @arch *
> +DESC
> @sha zVmYJtE6NhOnNIiaIO/Mg3dqWHUzbGSvEs4uCd9lAvQ=
> @size 602
> @depend devel/p5-Params-Util:p5-Params-Util-*:p5-Params-Util-1.07p2
> @cwd /usr/local
> libdata/perl5/site_perl/File/
> libdata/perl5/site_perl/File/PathList.pm
> @sha xPu9e1UGezN3CF1fdqjA9eWCPK0yALnGUj/3DA/UFjI=
> @size 5191
> @ts 1215675005
> @man man/man3p/File::PathList.3p
> @sha v3i5tb/ui1rMuD1cs5tt3H2RWq1mDmy1WGdc5B9cCUM=
> @size 6216
> @ts 1620344553
> 
> 
> I'm getting the following when doing doas make install:
> 
> CX p5-File-PathList $ MI
> doas (s...@mail.bennettconstruction.us) password: 
> ===>  Installing p5-File-PathList-1.04 from /usr/packages/amd64/all/
> doas (r...@mail.bennettconstruction.us) password: 
> quirks-4.9 signed on 2021-05-06T19:40:58Z
> quirks-4.6->4.9: ok
> file:/usr/packages/amd64/all/p5-File-PathList-1.04.tgz: unsigned package

I had a similar error, and fixed it by adding keepenv to my doas
config. I think the variable that needs to be passed through is
TRUSTED_PKG_PATH.

http://openbsd-archive.7691.n7.nabble.com/signify-error-when-installing-ports-on-current-td366895.html

-- 
James

Re: NEW: devel/git-annex

[James Cook]

On Sat, Mar 06, 2021 at 12:25:59PM -0800, Greg Steuck wrote:
> Greg Steuck  writes:
> >> I'm happy to prepare an 8.10-compatible version now if that would help
> >> with the upgrade to 8.10.
> >
> > Thanks, that wouldn't hurt :)
> 
> I gave it a whirl, what do you think:

Thanks. I just built the below version (with your ghc 8.10 update
patches, and also your patches from another email to remove ghc.port.mk
etc) and tried it out, and confirm that it works.

-- 
James

Re: NEW: devel/git-annex

[James Cook]

On Thu, Mar 04, 2021 at 09:21:18PM -0800, Greg Steuck wrote:
> Hi James,
> 
> Thanks for gathering the data and analyzing it!
> 
> I do not believe that speed of git-annex build should be a blocker to
> its inclusion into the tree. I'm just waiting for somebody to OK it
> before committing. Though maybe it's best to do that after the ghc 8.10
> upgrade, not sure?

I guess the only issue is MODCABAL_MANIFEST might need to be
re-generated? (I needed to do that when I rebased the port from your
ghc 8.10 branch onto master with ghc 8.6.)

I'm happy to prepare an 8.10-compatible version now if that would help
with the upgrade to 8.10.

-- 
James

Re: NEW: devel/git-annex

[James Cook]

> I propose we don't solve the problem until it's big enough to warrant
> our attention. I know I'm bad at predicting how things will develop a
> year from now. This is why I don't want to speculatively build more
> complexity.

Okay, makes sense!

> Thanks
> Greg
> 

-- 
James

Re: NEW: devel/git-annex

[James Cook]

> Full output attached.

Actually attached now.

-- 
James
815.882 1614811944.118 Building executable 'git-annex' for 
git-annex-8.20210223..
280.208 161481.486 Building library for aeson-1.5.6.0..
230.755 1614808139.685 Building library for cryptonite-0.28..
225.882 1614810601.156 Building library for statistics-0.15.2.0..
208.147 1614811298.421 Building library for persistent-sqlite-2.11.1.0..
185.04 1614811722.136 Building library for aws-0.22..
180.241 1614807599.353 Building library for vector-0.12.2.0..
154.123 1614810386.431 Building library for lens-5.0.1..
126.281 1614810903.624 Building library for persistent-2.11.0.2..
125.477 1614807999.721 Building library for language-javascript-0.7.1.0..
99.608 1614810283.687 Building library for tls-1.5.5..
83.839 1614806595.019 Building library for basement-0.0.11..
75.735 1614808382.587 Building library for blaze-html-0.9.1.2..
71.822 1614808706.039 Building library for attoparsec-0.13.2.5..
70.498 1614810830.043 Building library for shakespeare-2.0.25..
70.415 1614807797.865 Building library for regex-tdfa-1.3.1.0..
68.551 1614809082.604 Building library for mono-traversable-1.0.15.1..
64.685 1614811654.011 Building library for yesod-form-1.6.7..
62.772 1614811168.105 Building library for yesod-core-1.6.18.8..
55.405 1614811509.760 Building library for criterion-1.5.9.0..
53.642 1614811055.287 Building library for git-lfs-1.1.1..
52.201 1614809237.867 Building library for cassava-0.5.2.0..
51.79 1614808609.467 Building library for vector-algorithms-0.8.0.4..
45.046 1614808838.739 Building library for QuickCheck-2.14.2..
43.829 1614809007.377 Building library for http2-2.0.6..
39.643 1614809736.619 Building library for free-5.1.6..
39.511 1614809804.871 Building library for xml-conduit-1.9.0.0..
39.432 1614809153.893 Building library for mwc-random-0.15.0.1..
39.182 1614807169.388 Building library for tagsoup-0.14.8..
37.96 1614806856.055 Building executable 'happy' for happy-1.20.0..
33.759 1614807442.706 Building library for unordered-containers-0.2.13.0..
33.757 1614808663.909 Building library for math-functions-0.3.4.1..
33.397 1614809664.036 Building library for libyaml-0.1.2..
32.254 1614809508.294 Building library for http-api-data-0.4.3..
31.96 1614811263.194 Building library for persistent-template-2.9.1.0..
31.934 1614809449.611 Building library for conduit-1.3.4..
29.697 1614809921.767 Building library for feed-1.3.0.1..
29.08 1614809310.535 Building library for http-client-0.7.6..
28.587 1614809342.371 Building library for bifunctors-5.5.10..
28.497 1614810569.464 Building library for wai-extra-3.1.6..
26.171 1614809954.460 Building library for warp-3.3.14..
26.121 1614811234.037 Building library for xml-hamlet-0.5.0.1..
26.118 1614809407.861 Building library for tasty-1.4.1..
25.566 1614807053.668 Building library for primitive-0.7.1.0..
24.994 1614808897.884 Building library for skein-1.0.9.4..
24.75 1614806731.289 Building library for cereal-0.5.8.1..
24.151 1614806515.347 Building executable 'alex' for alex-3.2.6..
23.887 1614809543.364 Building library for semigroupoids-5.3.5..
23.823 1614811910.510 Building library for yesod-1.6.1.0..
23.766 1614806905.618 Building library for hourglass-0.2.12..
22.691 1614810543.692 Building library for yaml-0.11.5.0..
20.873 1614811588.200 Building library for yesod-static-1.6.1.0..
20.601 1614809604.179 Building library for x509-1.7.5..
20.536 1614807870.804 Building library for random-1.2.0..
20.527 1614809570.023 Building library for profunctors-5.6.2..
19.865 1614806485.257 Building library for SafeSemaphore-0.10.1..
19.715 1614808789.894 Building library for streaming-commons-0.2.2.1..
19.342 1614806931.626 Building executable 'hsc2hs' for hsc2hs-0.68.7..
19.304 1614811032.984 Building library for microstache-1.0.1.2..
18.767 1614808487.666 Building library for unliftio-0.2.14..
18.731 1614807490.553 Building library for psqueues-0.2.7.2..
18.181 161481.911 Building library for criterion-measurement-0.1.2.0..
18.166 1614807331.592 Building library for base-compat-batteries-0.11.2..
17.963 1614806551.667 Building library for base-compat-0.11.2..
17.696 1614808980.777 Building library for optparse-applicative-0.16.1.0..
17.691 1614807369.938 Building library for memory-0.15.0..
17.573 1614807548.253 Building library for network-3.1.0.1..
17.267 1614809700.315 Building library for conduit-extra-1.3.5..
17.111 1614807900.089 Building library for crypto-api-0.13.3..
16.385 1614806992.352 Building library for mime-types-0.1.0.9..
15.689 1614808560.700 Building library for iproute-1.7.10..
15.308 1614807210.973 Building library for th-abstraction-0.4.2.0..
14.924 1614811149.882 Building library for wai-app-static-3.1.7.2..
14.894 1614807352.232 Building library for time-compat-1.9.5..
14.859 1614808518.036 Building library for concurrent-output-1.10.12..
14.851 1614807285.174 Building library for xml-types-0.3.8..
14.568 1614808535.503 Building library for socks-0.6.1..
14.258 1614807302.935 

Re: NEW: devel/git-annex

[James Cook]

> This seems to me like a reasonable port to have. My only concern is it
> takes 40 minutes to build with MAKE_JOBS=4 on a 4 core machine:
> cpu0: Intel(R) Core(TM) i7-10610U CPU @ 1.80GHz, 7097.97 MHz, 06-8e-0c

I don't have an easy solution to that, but here are some data and a
couple of thoughts:


# Data

On my slower machine, without MAKE_JOBS*, here's how the time to build
devel/git-annex is divided among different Cabal packages:

- Total: 1h45m
- 14m (13%) building git-annex itself
- 4.7m (4.5%) building aeson
- 3.9m (3.7%) building cryptonite
- 3.5m (3.3%) building persistent-sqlite
- 2-3min each (~12.2%) building aws, vector, lens, persistent,
  language-javascript
- The rest (~63%): A couple hundred faster packages.

I measured this by building with
MODCABAL_BUILD_ARGS=--verbose=normal+timestamp, artificially adding one
more timestamp at the end, then running the output through
egrep '^[0-9.]+ ' build.log | tail -r | awk '{ print n-$1, $0; n = $1 }' |
sort -nr. Full output attached.

*I used the default MAKE_JOBS=1 because Cabal wasn't giving "completed"
 timestamps and I can't see how else to measure the time. The fraction
 of time used for git-annex itself presumably increases somewhat when
 the build is parallel.


# Thoughts

I agree with Greg's comment in the other thread that the ideal
solutions would be speeding up ghc or something like ccache. But I
still wonder if there's a quick-and-dirty way to get a modest speedup
without too much maintenance cost.

Earlier I suggested creating ports for a handful of slow packages, to
speed up builds. The numbers above show that's won't help much, unless
"handful" means hundreds and then we're back where we started.

What about adding a single port with a "collection" of commonly used
Hackage libraries? print/texlive does this with latex packages. If we
added a port with the few hundred most-used hackage packages, and other
Haskell packages could start with that, I'm guessing that would cut
down the total build time a lot, at the cost of having one big slow
port.

If there's any interest I could probably find some time to try
implementing it. I don't know how hard it would be.


-- 
James

NEW: devel/git-annex

[James Cook]

Greg Steuck wrote much of this --- see the earlier thread "Question
about lang/ghc module (trying to port git-annex)".

Now that Greg's new cabal infrastructure is checked in this should be
ready. I've been using a previous version for a while, and briefly
tested the attached version, which includes a few changes:

* Update to latest release, and remove the patches, which are now
  included.
* Move devel/libmagic to LIB_DEPENDS.
* Remove TEST_DEPENDS. ("make test" is not implemented.)
* Edit COMMENT.
* Make portcheck happy by wrapping >80char line and adding \n at end of
  DESCR.

Attached, or see
https://github.com/falsifian/ports/commit/96a4ca841691f785153f85c24e816b2a86f8f2c3
and the parent commit.

-- 
James


git-annex.tar.gz
Description: Binary data

Re: [PATCH 5/7] Use devel/cabal module for devel/darcs and upgrade to 2.16.3

[James Cook]

On Thu, Feb 25, 2021 at 08:16:43PM -0800, Greg Steuck wrote:
> From 3d80a96ac1803a72ea1fe7ab6ac28b5f79aad684 Mon Sep 17 00:00:00 2001
> From: Greg Steuck 
> Date: Mon, 24 Aug 2020 19:23:42 -0700
> Subject: [PATCH 5/7] Use devel/cabal module for devel/darcs and upgrade to
>  2.16.3

I gave this version a quick try and it seems to work; thanks.

-- 
James

Re: Question about lang/ghc module (trying to port git-annex)

[James Cook]

> > A few notes / questions:
> >
> > * I fixed a couple of typos in
> >   
> > https://github.com/falsifian/ports/commit/98111e702634842865a3d27675b6848e3fb2b7ca
> >
> > * I added example usage for cabal-bundler. I hope I got it right:
> >   
> > https://github.com/falsifian/ports/commit/f21cb097f215831b1d08443f9074073895a13731
> 
> Neat. I'll pick these up.
> 
> > * If I find more time to work on this, should I polish my git-annex
> >   port, or is there something more useful I could be doing with your
> >   Cabal infrastructure? (I installed darcs and it seems to work.)
> 
> Great! I don't think there's much else to do besides waiting for
> cabal-install 3.4.0.0 to be pushed to Hackage. Depending on how this
> aligns with 6.9 release we may or may not ship it then.
> 
> You could try to build your own port of a Haskell binary you use that we
> don't have yet. Your experience will likely tell us something about the
> level of maturity of this infra.

The only unported Haskell binaries I use are personal projects that are
not on Hackage. I could try locally porting one of those as an
experiment.

> > * I tried to enable tests, partly because Darcs has a comprehensive
> >   test suite. I ran into some trouble. My WIP is here:
> >   
> > https://github.com/falsifian/ports/commit/99cb0e33e2263cf15979a19bf068d345630c
> >   One problem is that cabal-bundler doesn't seem to be outputting test
> >   dependencies.
> 
> I keep track of this issue in https://github.com/blackgnezdo/ports/issues/13.
> See if you agree with my assessment there.

Thanks, I commented there.

> > * It's a pity that your approach doesn't allow Cabal-based ports to
> >   share dependencies.
> 
> I chose to trade-off compute to gain simlpicity and flexibility:
> https://github.com/blackgnezdo/ports/issues/3#issuecomment-650823377
> 
> Until this choice turns to be expensive enough to cause port builders to
> complain I'm inclined to keep this approach.

Okay, that's reasonable. My only concern: later on if people do
complain and it's decided it's a real problem, is there a path forward
that builds on what you've already got? Or would you have to start from
scratch to address it?

(I could complain already, but for me at least your solution is a
definite improvement over the status quo. The build times aren't the
end of the world.)

> > Some thoughts on mitigating or solving this, in decreasing order of 
> > plausability:
> >   * Is it possible to hand-pick some particularly slow ones, make
> > packages for them, and then tell Cabal to use those packages? E.g.
> > aeson takes a while to build.
> 
> Sadly this will move towards tree-wide configuration management which
> was a very non-fun thing to do as you experienced first-hand trying to
> deal with a ton of ports when you tried to add git-annex.

Maybe there could be a rule of thumb, like adding a reusable port is
only worth it if it reduces the total time to build all ports by at
least X hours?

I don't know if there are any libraries that meet that definition for
an appropriate value of X.

> > I guess it would be ugly to fill the ports tree with hundreds(?) of
> > directories each with an auto-generated package. I'm not sure if
> > there's a way to smoosh it all into one big file. E.g. I see
> > there's something called "multi-packages", but I'm guessing that's
> > not designed to handle hundreds of subpackages.
> 
> I've seen how these things play out in much bigger code bases with many
> people over time. We don't have enough people working on ports@ to
> manage this.

That makes sense. I imagine your approach will keep things easier for
maintainers: e.g. I'm not going to accidentally break darcs by updating
the dependencies of git-annex.

> >   * Is it possible to cheat, by having all the Cabal-based ports share
> > their Cabal directory? So if I build darcs then git-annex, Cabal
> > has access to all the build products from darcs when it builds
> > git-annex. I guess one problem is this might make the build output
> > depend on the order in which you build things, if Cabal
> > opportunistically uses whatever versions are there.
> 
> The root of the problem is GHC compilation speed. My preferred and
> universally useful optimizations would be:
> 
>   * Use/build something like ccache for ghc
>   * Speed up ghc itself

Those would be cool!

> > * Are you planning to commit these changes all at once? Or is there
> >   some way to have an intermediate state where both old-style and
> >   new-style packages work?
> 
> I don't see how it would be possible to have the old and new coexist.
> ghc-8.10 is incompatible with the haskell ports in the tree. My plan is
> to land this as a bundle. I wrote it up in an email to ports@ before.

I didn't know about the incompatibility. Maybe there could be two
separate ghc versions for a while, but I guess that's more trouble than
it's worth.

> Thanks
> Greg

-- 
James

Re: Question about lang/ghc module (trying to port git-annex)

[James Cook]

Hi Greg,

> >> I'll dig into this a bit. I simply didn't need to worry about this up
> >> until now as all the other ports don't do much custom Setup.hs work.
> >
> > I don't know if there will be any cases other than man pages and
> > git-annex's command aliases. Not sure how much effort it's worth.
> 
> The default action of doing nothing is valid :)

Incidentally, I asked the maintainer how man pages are supposed to be
installed, and they said it's best to use the Makefile if you want a
complete installation. Probably not worth changing our port, but
something to keep in mind in case this turns out to be a pattern.
https://git-annex.branchable.com/bugs/__91__Patch__93___fix___34__mdwn2man__58___cannot_execute_-_...__34__/


A few notes / questions:

* I fixed a couple of typos in
  
https://github.com/falsifian/ports/commit/98111e702634842865a3d27675b6848e3fb2b7ca

* I added example usage for cabal-bundler. I hope I got it right:
  
https://github.com/falsifian/ports/commit/f21cb097f215831b1d08443f9074073895a13731

* If I find more time to work on this, should I polish my git-annex
  port, or is there something more useful I could be doing with your
  Cabal infrastructure? (I installed darcs and it seems to work.)

* I tried to enable tests, partly because Darcs has a comprehensive
  test suite. I ran into some trouble. My WIP is here:
  
https://github.com/falsifian/ports/commit/99cb0e33e2263cf15979a19bf068d345630c
  One problem is that cabal-bundler doesn't seem to be outputting test
  dependencies. (I tried manually adding some, not included in my
  commit.) Another is that, even if I add test dependencies manually, I
  end up with strange results like this:

falsifian moth darcs $ make test
===>  Regression tests for darcs-2.16.2
cd /usr/local/ports/pobj/darcs-2.16.2/darcs-2.16.2  && /usr/bin/env -i 
PORTSDIR="/usr/ports" LIBTOOL="/usr/bin/libtool"  
PATH='/usr/local/ports/pobj/darcs-2.16.2/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11R6/bin'
 PREFIX='/usr/local'  LOCALBASE='/usr/local' X11BASE='/usr/X11R6'  CFLAGS='-O2 
-pipe'  TRUEPREFIX='/usr/local' DESTDIR=''  HOME='/darcs-2.16.2_writes_to_HOME' 
PICFLAG="-fpic"  BINGRP=bin BINOWN=root BINMODE=755 NONBINMODE=644  DIRMODE=755 
 INSTALL_COPY=-c INSTALL_STRIP=-s  MANGRP=bin MANOWN=root MANMODE=644 
BSD_INSTALL_PROGRAM="/usr/local/ports/pobj/darcs-2.16.2/bin/install -c -s -m 
755"  BSD_INSTALL_SCRIPT="/usr/local/ports/pobj/darcs-2.16.2/bin/install -c -m 
755"  BSD_INSTALL_DATA="/usr/local/ports/pobj/darcs-2.16.2/bin/install -c -m 
644"  BSD_INSTALL_MAN="/usr/local/ports/pobj/darcs-2.16.2/bin/install -c -m 
644"  BSD_INSTALL_PROGRAM_DIR="/usr/local/ports/pobj/darcs-2.16.2/bin/install 
-d -m 755"  
BSD_INSTALL_SCRIPT_DIR="/usr/local/ports/pobj/darcs-2.16.2/bin/install -d -m 
755"  BSD_INSTALL_DATA_DIR="/usr/local/ports/pobj/darcs-2.16.2/bin/install -d 
-m 755"  BSD_INSTALL_MAN_DIR="/usr/local/ports/pobj/darcs-2.16.2/bin/install -d 
-m 755" HOME=/usr/local/ports/pobj/darcs-2.16.2 /usr/local/bin/cabal v2-test 
--offline --disable-benchmarks  -w /usr/local/bin/ghc  -j1  --flags="curl 
-library"
Warning: No remote package servers have been specified. Usually you would 
have
one specified in the config file.
Resolving dependencies...
cabal: Could not resolve dependencies:
[__0] trying: random-1.2.0 (user goal)
[__1] trying: splitmix-0.1.0.3 (user goal)
[__2] trying: splitmix:!bench
[__3] rejecting: splitmix:*test (cyclic dependencies; conflict set: random,
splitmix)
[__1] fail (backjumping, conflict set: random, splitmix)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: random, splitmix

  splitmix's test target depends on random and random depends on
  splitmix. Normally this doesn't cause any problem; I don't know why
  it's a problem here.

* It's a pity that your approach doesn't allow Cabal-based ports to
  share dependencies. Some thoughts on mitigating or solving this, in
  decreasing order of plausability:

  * Is it possible to hand-pick some particularly slow ones, make
packages for them, and then tell Cabal to use those packages? E.g.
aeson takes a while to build.

  * nixpkgs (the only package system I'm really familiar with) deals
with Cabal by automatically generating package descriptions for
everything on hackage. I wonder if something along those lines
would be do-able.

I guess it would be ugly to fill the ports tree with hundreds(?) of
directories each with an auto-generated package. I'm not sure if
there's a way to smoosh it all into one big file. E.g. I see
there's something called "multi-packages", but I'm guessing that's
not designed to handle hundreds of subpackages.

  * Is it possible to cheat, by having all the Cabal-based ports share
their Cabal directory? So if I build darcs then git-annex, Cabal
has access to all the build products from 

Re: Question about lang/ghc module (trying to port git-annex)

[James Cook]

On Fri, Feb 05, 2021 at 09:57:48PM -0800, Greg Steuck wrote:
> James Cook  writes:
> 
> >> Would you like to try your hand in extending post-install target with
> >> some man formatting magic like we have in other ports?
> >
> > Done (commit a1c5aec8) in my "git-annex" branch:
> >
> > https://github.com/falsifian/ports/commits/git-annex
> 
> Very nice!
> 
> > It's not pretty. I replicated some logic from Build/Man.hs, which is
> > called from Setup.hs.
> 
> Some quick observations. Makefile allows line-continuation with \ and
> this is how people write long command lines. The for loop would be more
> readable if it were split like that. I also don't see a lot of places in
> ports make files with quotes around variables. I suspect we just expect
> people to not go nuts with spaces in directory names when building
> ports.

Thanks. I split up the long command and removed quotes:

https://github.com/falsifian/ports/commit/e24278e3c1d7fca1b3c4961a0e81a5bb2ca5bcf0

Two notes:

* Currently "make post-build" fails if you run it twice. Adding -p in
  "mkdir ${MAN1_STAGING_DIR}" would fix that. I don't know if it's
  better to fail or not if "make post-build" gets run more than once.

* I see you put "@" before the ln -s lines. I wasn't sure whether to
  add them to the other commands. The output is not that interesting.

> > Normally cabal v2-install would do that work for
> > us (putting the result in .cabal/store/ghc-XXX/git-annex-XXX) but I
> > don't know if it's worth getting that working.
> >
> > Caveat: I took a shortcut when testing this: instead of re-running
> > "make build" I just made the post-build target after adding this in.
> > I'll try building from scratch later if you don't beat me to it.
> 
> I believe `make rebuild` should reuse the cabal built pieces. post-build
> seems like a reasonable place for this kind of work. Alternatively
> sticking those man-formatting commands into post-install also seems
> acceptable to me.

I confirmed the port works by installing on a different machine.

One hitch: I needed to manually uninstall the current cabal-install
port because it conflicts with the new cabal port. Is there a way to
tell ports that cabal is a successor to cabal-install?

(I considered putting the man-building commands in post-install, but I
wasn't sure where to put the staging directory if I did that. I wanted
the staging directory because I assumed I should use the ${INSTALL_MAN}
macro if possible to put the files in their final place.)

> > On that branch I also removed the runtime dep on devel/git-lfs (it's
> > just one of a large number of optional backends).
> 
> Good call.
> 
> >> From what I gathered the v2-install target is largely unusable for
> >> installing packages outside of .cabal tree. At least neither I nor
> >> FreeBSD maintainer found a way to leverage that. Hence the manual
> >> install flow.
> >
> > I probably don't know all the subtleties, but when I run
> > "cabal v2-install", I get a nice set of files under
> > ~/.cabal/ghc-XXX/git-annex-XXX, including the man pages and all three
> > needed binaries. I would guess that just copying those files to the
> > destination would do the trick for most cabal executable packages, but
> > I haven't actually tried it. Maybe it's better to be cautious.
> 
> I'll dig into this a bit. I simply didn't need to worry about this up
> until now as all the other ports don't do much custom Setup.hs work.

I don't know if there will be any cases other than man pages and
git-annex's command aliases. Not sure how much effort it's worth.

> Thanks
> Greg

-- 
James

Re: Question about lang/ghc module (trying to port git-annex)

[James Cook]

> > From what I gathered the v2-install target is largely unusable for
> > installing packages outside of .cabal tree. At least neither I nor
> > FreeBSD maintainer found a way to leverage that. Hence the manual
> > install flow.
> 
> I probably don't know all the subtleties, but when I run
> "cabal v2-install", I get a nice set of files under
> ~/.cabal/ghc-XXX/git-annex-XXX, including the man pages and all three
> needed binaries. I would guess that just copying those files to the
> destination would do the trick for most cabal executable packages, but
> I haven't actually tried it. Maybe it's better to be cautious.

I mean ~/.cabal/store/ghc-XXX/git-annex-XXX.

Also forgot to mention the following git-annex patch (already reported
upstream) is needed if anyone does want to experiment with using cabal
v2-install to get git-annex man pages.

diff --git a/Build/Mans.hs b/Build/Mans.hs
index 9fb29d4a3..672dcd71c 100644
--- a/Build/Mans.hs
+++ b/Build/Mans.hs
@@ -38,7 +38,8 @@ buildMans = do
if (Just srcm > destm)
then do
r <- system $ unwords
-   [ "./Build/mdwn2man"
+   [ "perl"
+   , "Build/mdwn2man"
, progName src
, "1"
, src

Re: Question about lang/ghc module (trying to port git-annex)

[James Cook]

On Thu, Feb 04, 2021 at 11:13:43AM -0800, Greg Steuck wrote:
> James Cook  writes:
> 
> > I don't think that's an rsync problem. Unfortunately the
> > git-annex-shell binary is missing, and that's pretty critical (needed
> > to access the repository from other machines; sort of like git
> > push/pull).
> 
> This part is easy, patch attached.

Thanks, I can now access my git-annex repositories remotely without an
awful kludge.

> > Incidentally, the man pages also seem to be missing.
> 
> Would you like to try your hand in extending post-install target with
> some man formatting magic like we have in other ports?

Done (commit a1c5aec8) in my "git-annex" branch:

https://github.com/falsifian/ports/commits/git-annex

It's not pretty. I replicated some logic from Build/Man.hs, which is
called from Setup.hs. Normally cabal v2-install would do that work for
us (putting the result in .cabal/store/ghc-XXX/git-annex-XXX) but I
don't know if it's worth getting that working.

Caveat: I took a shortcut when testing this: instead of re-running
"make build" I just made the post-build target after adding this in.
I'll try building from scratch later if you don't beat me to it.

On that branch I also removed the runtime dep on devel/git-lfs (it's
just one of a large number of optional backends).

> > Alternatively, I think the cabal (v2-)install commands installs what's
> > needed (at least, it gets git-annex-shell, which is the really
> > important thing). Is there some way to take advantage of that in your
> > cabal infrastrsucture?
> 
> From what I gathered the v2-install target is largely unusable for
> installing packages outside of .cabal tree. At least neither I nor
> FreeBSD maintainer found a way to leverage that. Hence the manual
> install flow.

I probably don't know all the subtleties, but when I run
"cabal v2-install", I get a nice set of files under
~/.cabal/ghc-XXX/git-annex-XXX, including the man pages and all three
needed binaries. I would guess that just copying those files to the
destination would do the trick for most cabal executable packages, but
I haven't actually tried it. Maybe it's better to be cautious.

-- 
James

Re: Question about lang/ghc module (trying to port git-annex)

[James Cook]

> > Assuming it works and that "git annex test" doesn't show any problems,
> > is there anything else I can do to help with git-annex?
> 
> The idea to link the test code into the main binary is quite
> unorthodox. I can imagine some arguments for it, but I'll have to
> overcome a lot of prior beliefs to accept this as a best practice.

This is the only place I've seen it. Maybe the author did it this way
so that users can reassure themselves that their storage backends are
working (using the more specialized "git annex testremote" command).

> $ touch /home/greg/.config/git-annex/program
> $ git-annex test
> ...
> get (ssh remote): FAIL (1.20s)
>   ./Test/Framework.hs:57:
>   get of file failed (transcript follows)
>   get foo (from origin...) /bin/sh: git-annex-shell: not foundrsync: 
> connection unexpectedly closed (0 bytes received so far) [Receiver]rsync 
> error: error in rsync protocol data stream (code 12) at io.c(228) 
> [Receiver=3.2.3]rsync exited 12  rsync failed -- run git annex again to 
> resume file transfer  transfer failed  Unable to access these remotes: origin 
>  Try making some of these remotes available:  
> a45e9dcd-c958-4026-96e4-a0d9a6fc322b -- test repo [origin]failedgit-annex: 
> get: 1 failed
> ...
> 6 out of 975 tests failed (1002.59s)
>   (Failures above could be due to a bug in git-annex, or an incompatibility
>with utilities, such as git, installed on this system.)
> 
> The failing tests are due to unconfigured rsync stuff.

I don't think that's an rsync problem. Unfortunately the
git-annex-shell binary is missing, and that's pretty critical (needed
to access the repository from other machines; sort of like git
push/pull). Incidentally, the man pages also seem to be missing.

(I think git-annex tries to connect a local rsync process to a remote
git-annex-shell process, which would explain why we also see an rsync
error message.)

I guess this could be solved with custom logic in
devel/git-annex/Makefile. I can try to write some custom install
commands, maybe tomorrow or Friday. (If you point me to the appropriate
variable for overriding/augmenting the install script, that would help;
I'm still pretty new to ports.)

Alternatively, I think the cabal (v2-)install commands installs what's
needed (at least, it gets git-annex-shell, which is the really
important thing). Is there some way to take advantage of that in your
cabal infrastrsucture?

-- 
James

Re: Ports documentation: document that TRUSTED_PKG_PATH is needed in doas.conf?

[James Cook]

On Wed, Feb 03, 2021 at 08:29:31PM -0700, Thomas Frohwein wrote:
> On Wed, Feb 03, 2021 at 09:23:48PM +0000, James Cook wrote:
> > On Wed, Feb 03, 2021 at 07:24:08PM +, Stuart Henderson wrote:
> > > On 2021/02/03 17:39, James Cook wrote:
> > > > Hi ports@,
> > > > 
> > > > Summary: I suggest the section at
> > > > https://www.openbsd.org/faq/ports/ports.html#PortsConfig
> > > > should include some additional text like the following:
> > > > 
> > > >   You will need to configure doas to pass the TRUSTED_PKG_PATH variable
> > > >   when running /usr/sbin/pkg_add. Adding the "nopass" option for
> > > >   certain commands can help reduce the number of times a password needs
> > > >   to be entered. For example, add the following to doas.conf(5),
> > > >   replacing "myuser" with your username:
> > > > 
> > > > permit nopass myuser cmd /usr/bin/touch
> > > > permit nopass setenv { TRUSTED_PKG_PATH TERM } myuser cmd 
> > > > /usr/sbin/pkg_add
> > > > permit nopass setenv { TERM } myuser cmd /usr/sbin/pkg_delete
> > > 
> > > Let's not go down the 'try to evaluate every variable' path again,
> > > keepenv is the way to go, we have spent much time figuring out weird
> > > bugs from missing variables in the past when we have tried to
> > > evaluate them.
> > > 
> > > If you are going to allow pkg_add with "nopass" you might just as well
> > > write "permit nopass myuser". An account which can run pkg_add as root
> > > has full control of the system.
> > 
> > I don't have strong opinions about that. My point is just that the
> > current documentation left me with a setup that didn't work.
> > 
> > How about recommending keepenv instead, if that's better?
> 
> I think sthen@ may have been a little too subtle about what a giant
> footgun your proposal is.
> 
> "permit nopass myuser" is equivalent to myuser being root and you might
> as well run everything as root then and toss out all security
> considerations that come from logging in as a non-root user.
> 
> This has no place in the FAQ in my opinion.

I'll readily accept that it's a bad solution. I just thought I should
suggest something instead of just complaining that the documentation is
broken.

Maybe I should back up and explain what led me to believe the
documentation is broken. Maybe I'm mistaken, and the problem I ran into
is my own fault.

* The phrasing in this section:
  https://www.openbsd.org/faq/ports/ports.html#PortsConfig led me to
  believe I should try to configure privilege separation if I want to
  work with ports.

* As a result, I set PORTS_PRIVSEP=Yes in /etc/mk.conf.

* With that configuration, running "make install" in, say
  /usr/ports/archivers/gtar results in an error:

falsifian angel gtar $ make install
doas (falsif...@angel.falsifian.org) password:
doas (falsif...@angel.falsifian.org) password:
doas (falsif...@angel.falsifian.org) password:
===>  Installing gtar-1.33p0 from /usr/ports/packages/amd64/all/
doas (falsif...@angel.falsifian.org) password:
quirks-3.534 signed on 2021-02-03T00:42:33Z
file:/usr/ports/packages/amd64/all/gtar-1.33p0.tgz: unsigned package
Can't find /usr/ports/packages/amd64/all/gtar-1.33p0.tgz
Couldn't install gtar-1.33p0
*** Error 1 in . (/usr/ports/infrastructure/mk/bsd.port.mk:2148 
'/var/db/pkg/gtar-1.33p0/+CONTENTS': @/usr/bin/env -i PKG_TMPDIR=/var/tmp TE...)
doas (falsif...@angel.falsifian.org) password:
*** Error 2 in /usr/ports/archivers/gtar 
(/usr/ports/infrastructure/mk/bsd.port.mk:2591 'install': @lock=gtar-1.33p0;  
export _LOCKS_HELD=" ...)

Of course, now I know how to solve the error. But I thought I'd found a
bug in the documentation, because the documentation led me to a bad
setup.

My suggested doas.conf comes (almost) directly from the existing
documentation for PORTS_PRIVSEP.

-- 
James

Re: Ports documentation: document that TRUSTED_PKG_PATH is needed in doas.conf?

[James Cook]

On Wed, Feb 03, 2021 at 07:24:08PM +, Stuart Henderson wrote:
> On 2021/02/03 17:39, James Cook wrote:
> > Hi ports@,
> > 
> > Summary: I suggest the section at
> > https://www.openbsd.org/faq/ports/ports.html#PortsConfig
> > should include some additional text like the following:
> > 
> >   You will need to configure doas to pass the TRUSTED_PKG_PATH variable
> >   when running /usr/sbin/pkg_add. Adding the "nopass" option for
> >   certain commands can help reduce the number of times a password needs
> >   to be entered. For example, add the following to doas.conf(5),
> >   replacing "myuser" with your username:
> > 
> > permit nopass myuser cmd /usr/bin/touch
> > permit nopass setenv { TRUSTED_PKG_PATH TERM } myuser cmd 
> > /usr/sbin/pkg_add
> > permit nopass setenv { TERM } myuser cmd /usr/sbin/pkg_delete
> 
> Let's not go down the 'try to evaluate every variable' path again,
> keepenv is the way to go, we have spent much time figuring out weird
> bugs from missing variables in the past when we have tried to
> evaluate them.
> 
> If you are going to allow pkg_add with "nopass" you might just as well
> write "permit nopass myuser". An account which can run pkg_add as root
> has full control of the system.

I don't have strong opinions about that. My point is just that the
current documentation left me with a setup that didn't work.

How about recommending keepenv instead, if that's better?

-- 
James

Re: Question about lang/ghc module (trying to port git-annex)

[James Cook]

On Sun, Jan 31, 2021 at 11:58:40PM -0800, Greg Steuck wrote:
> Greg Steuck  writes:
> 
> >> git-annex has at least 50 dependencies I couldn't find in ports, so I
> >> want to make sure I understand this before I start porting them one at
> >> a time (or just give up)...
> >
> > This is exactly the problem I want to solve with cabal.port.mk. You
> > can try to look at
> > https://marc.info/?l=openbsd-ports=160858285410366=2
> > A quick search in the archives will show the justification and the
> > history of the effort.
> >
> > The current state of the work is I'm waiting on cabal 3.4 official
> > release gated by ghc-9.0. I'm also looking for people needing this work
> > and your attempt to add git-annex gives me more motivation to finish.
> 
> FWIW, a version of git-annex port that can display its help message:
> https://github.com/openbsd/ports/commit/0d970c085441b82f0080afee13e2034e47f475c0

Thanks. I'm currently building it and will try it with my repos.

In devel/cabal/Makefile would it make sense to fix the ghc version,
i.e:

  BUILD_DEPENDS +=  lang/ghc=8.10.3

I quickly discovered cabal won't build with 8.6.4, which is what I have
right now, because the port expects ghc-prim-0.6.1 which doesn't match.
With the above change, "make build" kicks off a build for ghc-8.10.3
(which I'm now waiting for).

> I didn't bother to pare down the dependency list to exclude the test
> stuff, so MODCABAL_MANIFEST is twice the length it should be.

Hm, I thought git-annex doesn't have test stuff. cabal v2-test doesn't
work; the way to test it is to build the binary and run git annex test.

Assuming it works and that "git annex test" doesn't show any problems,
is there anything else I can do to help with git-annex? I use darcs a
fair bit; would it help if I built it on your branch and tested it out?

> Thanks
> Greg

-- 
James

Ports documentation: document that TRUSTED_PKG_PATH is needed in doas.conf?

[James Cook]

Hi ports@,

Summary: I suggest the section at
https://www.openbsd.org/faq/ports/ports.html#PortsConfig
should include some additional text like the following:

  You will need to configure doas to pass the TRUSTED_PKG_PATH variable
  when running /usr/sbin/pkg_add. Adding the "nopass" option for
  certain commands can help reduce the number of times a password needs
  to be entered. For example, add the following to doas.conf(5),
  replacing "myuser" with your username:

permit nopass myuser cmd /usr/bin/touch
permit nopass setenv { TRUSTED_PKG_PATH TERM } myuser cmd /usr/sbin/pkg_add
permit nopass setenv { TERM } myuser cmd /usr/sbin/pkg_delete

and also updating the section on the PORTS_PRIVSEP variable in
bsd.ports.mk(5) to replace

   If the regular user is not allowed to run privileged commands
   without entering a password, you may want these additional rules
   in doas.conf(5), to reduce the amount of times the password needs
   to be entered during ports work:

with

  You will need to configure doas to pass the TRUSTED_PKG_PATH variable
  when the regular user runs /usr/sbin/pkg_add. You can also reduce the
  number of times the password needs to be entered by permitting
  certain commands without a password. For example:

Happy to turn this into a patch if it looks good.


Reasoning:

I'm surprised this requirement isn't documented. "make install" as
non-root fails if TRUSTED_PKG_PATH isn't set. Am I missing something?
Or does every new ports user run into this problem, and quietly figure
out the solution on their own? Or do people just run everything as
root?

I just fixed the problem for myself after scratching my head for a
while and finally finding this email thread:
http://openbsd-archive.7691.n7.nabble.com/signify-error-when-installing-ports-on-current-td366895.html

To be fair, I did see the documentation for the PORTS_PRIVSEP variable,
which an example with TRUSTED_PKG_PATH. But I didn't add the suggested
lines, because the phrasing implies it's not actually needed:
"you may want these additional rules ...".

-- 
James

Re: Question about lang/ghc module (trying to port git-annex)

[James Cook]

> Here is what I have so far.

P.S. to anyone trying to build git-annex, you'll need the following
patch, which I didn't yet get around to adding to my WIP port:

https://git-annex.branchable.com/bugs/__91__PATCH__93___OpenBSD__58___fix_Utility.DirWatcher.Kqueue/?updated#comment-0b3828545e6cf6ec417c0f82645888cb

-- 
James

Question about lang/ghc module (trying to port git-annex)

[James Cook]

Hi ports@,

I'm trying to write a port for git-annex, which is built using
Haskell's Cabal. I'm new to OpenBSD porting and don't completely
understand the lang/ghc module.

Basic question: do all of the Haskell dependencies (from git-annex.cabal) need
to already have their own ports (like devel/hs-async, devel/hs-network, etc)?
(Or will the missing ones be magically built like cabal-install normally does?)
If they do need to all be ported, how isn't there e.g. a devel/hs-containers
port --- doesn't basically every Haskell project depend on the containers
package?

git-annex has at least 50 dependencies I couldn't find in ports, so I
want to make sure I understand this before I start porting them one at
a time (or just give up)...

Also, I'm using devel/darcs/Makefile as my model but don't understand
the following comment. Should I also be listing git-annex's
dependencies in BUILD_DEPENDS instead of LIB_DEPENDS?

# Yes, build dependencies, because GHC libs are still static and we
# don't want to pull in all of ghc.
BUILD_DEPENDS = archivers/hs-zip-archive>=0.2.3,<0.4 \
archivers/hs-zlib>=0.6.1.2,<0.7.0.0 \
...

Another question: what is MODGHC_PACKAGE_KEY? ghc.port.mk describes it
as the "package key" of the library, but I don't know what that means.

Here is what I have so far. I only included the dependencies that are
already in ports. Currently make build fails with a compiler error
"Could not find module ‘System.FilePath.ByteString’", indicating that
it is indeed a problem that there's no filepath-bytestring port.

diff --git a/devel/git-annex/Makefile b/devel/git-annex/Makefile
new file mode 100644
index 000..57ae7fe1079
--- /dev/null
+++ b/devel/git-annex/Makefile
@@ -0,0 +1,54 @@
+# $OpenBSD$
+#
+# TODO:
+# Use /usr/ports/infrastructure/bin/portcheck
+
+COMMENT =  manage files with git without checking in file contents
+
+DISTNAME =  git-annex-8.20210127
+CATEGORIES =   devel
+HOMEPAGE = https://git-annex.branchable.com/
+
+MODULES =   lang/ghc
+MODGHC_BUILD =  cabal hackage nort
+
+# Many licenses listed in COPYRIGHT. All permit redistribution.
+PERMIT_PACKAGE =   Yes
+
+# TODO
+# "make port-lib-depends-check" can help
+#WANTLIB = ???
+
+# TODO
+# Dependencies
+#BUILD_DEPENDS =   ???
+RUN_DEPENDS =  net/rsync devel/git-lfs
+LIB_DEPENDS =  devel/hs-network-uri>=2.6 \
+   devel/hs-exceptions>=0.6 \
+   devel/hs-data-default \
+   devel/hs-random \
+   devel/hs-dlist \
+   devel/hs-unix-compat \
+   devel/hs-async \
+   devel/hs-hslogger \
+   devel/hs-utf8-string \
+   devel/hs-sandi \
+   devel/hs-monad-control \
+   devel/hs-edit-distance \
+   devel/hs-resourcet \
+   devel/hs-conduit \
+   devel/hs-old-locale \
+   devel/hs-unliftio-core \
+   devel/hs-vector \
+   devel/hs-unordered-containers \
+   devel/hs-regex-tdfa \
+   devel/hs-byteable \
+   security/hs-crypto-api \
+   devel/hs-split \
+   textproc/hs-attoparsec \
+   devel/hs-QuickCheck
+#version doesn't match
+#devel/hs-network>=3.0.0.0
+TEST_DEPENDS = net/rsync
+
+.include 
diff --git a/devel/git-annex/distinfo b/devel/git-annex/distinfo
new file mode 100644
index 000..9d9fa1c5788
--- /dev/null
+++ b/devel/git-annex/distinfo
@@ -0,0 +1,2 @@
+SHA256 (ghc/git-annex-8.20210127.tar.gz) = 
Y29DlCDyipKoJQufi0IlZ+Q5MV8/LSPLC+o7Cg5XVcM=
+SIZE (ghc/git-annex-8.20210127.tar.gz) = 1361993

Re: collision in python-3.8.6 when upgrading 6.7 to snapshot

[James Cook]

On Mon, Jan 25, 2021 at 01:48:07PM +, Stuart Henderson wrote:
> On 2021/01/25 03:34, James Cook wrote:
> > On Thu, Oct 08, 2020 at 11:42:53AM +0100, Stuart Henderson wrote:
> > > On 2020/10/08 02:33, James Cook wrote:
> > > > I'm reporting a slight hiccup upgrading from 6.7 to snapshot. This might
> > > > count as my own fault, but reporting it in case it's something that's
> > > > supposed to run smoothly. (I was able to recover; no help needed.)
> > > > 
> > > > I upgraded from 6.7 to a snapshot (with sysupgrade -s), then ran
> > > > "doas pkg_add -u", and got this:
> > > > 
> > > > 
> > > > Collision in python-3.8.6: the following files already exist
> > > > /usr/local/bin/2to3 from python-3.8.6
> > > > /usr/local/bin/pydoc3 from python-3.8.6
> > > > /usr/local/bin/python3 from python-3.8.6
> > > > /usr/local/bin/python3-config from python-3.8.6
> > > > /usr/local/lib/pkgconfig/python3.pc from python-3.8.6
> > > > It seems to be a missing package registration
> > > 
> > > This is because,
> > > 
> > > 1) The default Python version in ports was switched from 3.7 to 3.8,
> > > so the symlinks for the "default names" moved at that time
> > > 
> > > 2) After this, Python 3.7 was updated in 6.7-stable
> > > 
> > > The @conflict lines in the PLIST files for 3.8 in -current should have
> > > been updated after the -stable update, so pkg_add doesn't know that
> > > it needs to update 3.7.9 to the "6.8 version" (without the symlinks)
> > > before installing 3.8.
> > > 
> > > We can't do anything for the main 6.8 release packages now, but it can
> > > be fixed in 6.8-stable once it's open. For those arches with -stable
> > > packages available, once those packages are built it shouldn't be
> > > noticeable to users.
> > > 
> > > > After this, pkg_delete partial-python; pkg_add python3; pkg_add -u
> > > > completed the upgrade without trouble (though I haven't tested 
> > > > anything).
> > > 
> > > Yes that should do the trick.
> > 
> > Somehow, this old problem is still haunting my computer. I'm reviving
> > this old thread for two reasons:
> > 
> > 1. In case the below is a useful bug report.
> > 
> > 2. To request advice dealing with it. (My goal: I'd like to have a
> >functioning /usr/local/bin/python3 symlink, and no
> >"partial-python-3.8.7" package.)
> > 
> > 
> > 
> > Ever since the problem I reported above, I've had trouble with the
> > Python3 symlinks whenever I run pkg_add -u on that same system. I
> > haven't kept careful track, so I'll start with the most recent
> > symptoms:
> > 
> > * Today, I noticed that I had no /usr/local/bin/python3 symlink, even
> >   though I have python-3.8.7 installed. (This is probably because I had
> >   earlier uninstalled the "partial-python-3.(something)" package.)
> > 
> > * Seeing that, I decided to force-reinstall Python 3.8, resulting in
> >   the following:
> > 
> >   falsifian angel tmp $ doas pkg_add -u -D installed python-3.8.7
> >   quirks-3.519 signed on 2021-01-24T21:22:51Z
> >   quirks-3.519->3.519: ok
> >   python-3.8.7:libffi-3.3->3.3: ok
> >   python-3.8.7:libiconv-1.16p0->1.16p0: ok
> >   python-3.8.7:gettext-runtime-0.21p0->0.21p0: ok
> >   python-3.8.7:sqlite3-3.34.0->3.34.0: ok
> >   python-3.8.7:xz-5.2.5->5.2.5: ok
> >   python-3.8.7:bzip2-1.0.8p0->1.0.8p0: ok
> >   Bogus symlink: /usr/local/bin/2to3
> >   Bogus symlink: /usr/local/bin/pydoc3
> >   Bogus symlink: /usr/local/bin/python3
> >   Bogus symlink: /usr/local/bin/python3-config
> >   Bogus symlink: /usr/local/lib/pkgconfig/python3.pc
> >   Bad rename /usr/local/bin/2to3 to /usr/local/bin/2to3.HZzyyvf4Qq: No such 
> > file or directory
> >   Bad rename /usr/local/bin/pydoc3 to /usr/local/bin/pydoc3.IA1AJiEcMU: No 
> > such file or directory
> >   Bad rename /usr/local/bin/python3 to /usr/local/bin/python3.puMIJ7N7oe: 
> > No such file or directory
> >   Bad rename /usr/local/bin/python3-config to 
> > /usr/local/bin/python3-config.QD8CNhk2EW: No such file or directory
> >   Bad rename /usr/local/lib/pkgconfig/python3.pc to 
> > /usr/local/lib/pkgconfig/python3.pc.jrFTvKCyzK: No such file or directory
> >   python-3.8.7->3.8.7: ok
> >   Read shared items: ok
>

Re: collision in python-3.8.6 when upgrading 6.7 to snapshot

[James Cook]

On Thu, Oct 08, 2020 at 11:42:53AM +0100, Stuart Henderson wrote:
> On 2020/10/08 02:33, James Cook wrote:
> > I'm reporting a slight hiccup upgrading from 6.7 to snapshot. This might
> > count as my own fault, but reporting it in case it's something that's
> > supposed to run smoothly. (I was able to recover; no help needed.)
> > 
> > I upgraded from 6.7 to a snapshot (with sysupgrade -s), then ran
> > "doas pkg_add -u", and got this:
> > 
> > 
> > Collision in python-3.8.6: the following files already exist
> > /usr/local/bin/2to3 from python-3.8.6
> > /usr/local/bin/pydoc3 from python-3.8.6
> > /usr/local/bin/python3 from python-3.8.6
> > /usr/local/bin/python3-config from python-3.8.6
> > /usr/local/lib/pkgconfig/python3.pc from python-3.8.6
> > It seems to be a missing package registration
> 
> This is because,
> 
> 1) The default Python version in ports was switched from 3.7 to 3.8,
> so the symlinks for the "default names" moved at that time
> 
> 2) After this, Python 3.7 was updated in 6.7-stable
> 
> The @conflict lines in the PLIST files for 3.8 in -current should have
> been updated after the -stable update, so pkg_add doesn't know that
> it needs to update 3.7.9 to the "6.8 version" (without the symlinks)
> before installing 3.8.
> 
> We can't do anything for the main 6.8 release packages now, but it can
> be fixed in 6.8-stable once it's open. For those arches with -stable
> packages available, once those packages are built it shouldn't be
> noticeable to users.
> 
> > After this, pkg_delete partial-python; pkg_add python3; pkg_add -u
> > completed the upgrade without trouble (though I haven't tested anything).
> 
> Yes that should do the trick.

Somehow, this old problem is still haunting my computer. I'm reviving
this old thread for two reasons:

1. In case the below is a useful bug report.

2. To request advice dealing with it. (My goal: I'd like to have a
   functioning /usr/local/bin/python3 symlink, and no
   "partial-python-3.8.7" package.)



Ever since the problem I reported above, I've had trouble with the
Python3 symlinks whenever I run pkg_add -u on that same system. I
haven't kept careful track, so I'll start with the most recent
symptoms:

* Today, I noticed that I had no /usr/local/bin/python3 symlink, even
  though I have python-3.8.7 installed. (This is probably because I had
  earlier uninstalled the "partial-python-3.(something)" package.)

* Seeing that, I decided to force-reinstall Python 3.8, resulting in
  the following:

  falsifian angel tmp $ doas pkg_add -u -D installed python-3.8.7
  quirks-3.519 signed on 2021-01-24T21:22:51Z
  quirks-3.519->3.519: ok
  python-3.8.7:libffi-3.3->3.3: ok
  python-3.8.7:libiconv-1.16p0->1.16p0: ok
  python-3.8.7:gettext-runtime-0.21p0->0.21p0: ok
  python-3.8.7:sqlite3-3.34.0->3.34.0: ok
  python-3.8.7:xz-5.2.5->5.2.5: ok
  python-3.8.7:bzip2-1.0.8p0->1.0.8p0: ok
  Bogus symlink: /usr/local/bin/2to3
  Bogus symlink: /usr/local/bin/pydoc3
  Bogus symlink: /usr/local/bin/python3
  Bogus symlink: /usr/local/bin/python3-config
  Bogus symlink: /usr/local/lib/pkgconfig/python3.pc
  Bad rename /usr/local/bin/2to3 to /usr/local/bin/2to3.HZzyyvf4Qq: No such 
file or directory
  Bad rename /usr/local/bin/pydoc3 to /usr/local/bin/pydoc3.IA1AJiEcMU: No such 
file or directory
  Bad rename /usr/local/bin/python3 to /usr/local/bin/python3.puMIJ7N7oe: No 
such file or directory
  Bad rename /usr/local/bin/python3-config to 
/usr/local/bin/python3-config.QD8CNhk2EW: No such file or directory
  Bad rename /usr/local/lib/pkgconfig/python3.pc to 
/usr/local/lib/pkgconfig/python3.pc.jrFTvKCyzK: No such file or directory
  python-3.8.7->3.8.7: ok
  Read shared items: ok
  --- -python-3.8.7 ---
  Files kept as partial-python-3.8.7 package

I have no idea what's happening here. Somehow, pkg_add has decided that
the symlinks (which didn't exist before I ran the command) should now
be part of a new "partial-python-3.8.7" package:

  falsifian angel tmp $ pkg_info -L partial-python-3.8.7
  Information for inst:partial-python-3.8.7
  
  Files:
  /usr/local/bin/2to3
  /usr/local/bin/pydoc3
  /usr/local/bin/python3
  /usr/local/bin/python3-config
  /usr/local/lib/pkgconfig/python3.pc

Here's what I remember from the time between the October 2020 thread
and today:

* For a while (possibly still continuing to today; I haven't been
  paying close attention) every time I ran pkg_add -u, I would see
  error messages related to /usr/local/bin/python3 and other symlinks
  (but the upgrade seems to work). I think the error messages were
  generally similar to what's shown above, but I'm not sure.

* The symlinks were generally kept in a "partial-python-3.(somet

Re: abiword segfaults immediately

[James Cook]

Thanks, Ed.

I just emailed the port maintainer separately to ask if they have any
ideas. I think it would be better if abiword didn't crash even if the
locale is unsupported.

FYI I tried setting LC_CTYPE to a supported value (en_US.UTF-8) while
keeping LANG at the unsupported en_CA.UTF-8 but that wasn't enough.

Admittedly I had read that FAQ entry and ignored the recommendation,
based on some wishful thinking. Now I will take it more seriously.

Off-topic: I'm curious if there's a clean way to convey my language
preference to software from ports, or I should just accept that en_US
is going to be my default. Not a big deal for en_CA, but what do
French/Japanese/etc speakers do?

On Mon, Dec 14, 2020 at 02:36:57PM +, Ed Gray wrote:
> Hi James,
> 
> I've noticed that in openbsd utf-8 is not universally supported like on
> other open source systems.
> 
> As far as I know you have to use en_US.UTF-8 or C for LC_CTYPE and I'm not
> sure if the LANG environment variable is used either.
> 
> Whether it should segfault in this case I don't know.
> 
> Perhaps once you are running abiword it will have its own native language
> support for documents and you won't need to set the LANG variable?
> 
> See this FAQ:
> 
> https://www.openbsd.org/faq/faq8.html
> 
> Regards
> Ed Gray
> 
> On Sun, 13 Dec 2020, 11:03 pm James Cook,  wrote:
> 
> > On Thu, Dec 03, 2020 at 04:48:04PM +, James Cook wrote:
> > > I just installed abiword and also just upgraded to the latest snapshot
> > > (and ran pkg_add -u). abiword immediately crashes when I run it. I have
> > > never tried running abiword on OpenBSD before. Is it broken for others
> > > or just for me?
> >
> > I just discovered it works if I set LANG to en_US.UTF-8 (in ksh,
> > LANG=en_US.UTF-8 abiword). Normally I have LANG set to en_CA.UTF-8.
> > Similar behaviour was reported in 2015:
> >
> > https://marc.info/?l=openbsd-ports=143301697319210=2
> >
> > This works as a workaround, but I wonder what the problem is.
> >
> > --
> > James

-- 
James

Re: abiword segfaults immediately

[James Cook]

On Thu, Dec 03, 2020 at 04:48:04PM +, James Cook wrote:
> I just installed abiword and also just upgraded to the latest snapshot
> (and ran pkg_add -u). abiword immediately crashes when I run it. I have
> never tried running abiword on OpenBSD before. Is it broken for others
> or just for me?

I just discovered it works if I set LANG to en_US.UTF-8 (in ksh,
LANG=en_US.UTF-8 abiword). Normally I have LANG set to en_CA.UTF-8.
Similar behaviour was reported in 2015:

https://marc.info/?l=openbsd-ports=143301697319210=2

This works as a workaround, but I wonder what the problem is.

-- 
James

Re: abiword segfaults immediately

[James Cook]

On Thu, Dec 03, 2020 at 04:48:04PM +, James Cook wrote:
> I just installed abiword and also just upgraded to the latest snapshot
> (and ran pkg_add -u). abiword immediately crashes when I run it. I have
> never tried running abiword on OpenBSD before. Is it broken for others
> or just for me?

Sorry, should probably include uname:

falsifian angel ~ $ uname -a
OpenBSD angel.falsifian.org 6.8 GENERIC.MP#206 amd64

and here's dmesg output in case it's helpful.

OpenBSD 6.8-current (GENERIC.MP) #206: Wed Dec  2 17:47:17 MST 2020
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 17161912320 (16366MB)
avail mem = 16626487296 (15856MB)
random: good seed from bootblocks
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.4 @ 0xf0100 (59 entries)
bios0: vendor Award Software International, Inc. version "F7" date 11/20/2009
bios0: Gigabyte Technology Co., Ltd. GA-MA790XT-UD4P
acpi0 at bios0: ACPI 1.0
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP SSDT HPET MCFG TAMG APIC
acpi0: wakeup devices PCI0(S5) USB0(S3) USB1(S3) USB2(S3) USB3(S3) USB4(S3) 
USB5(S3) USB6(S3) SBAZ(S4) P2P_(S5) PCE2(S4) PCE3(S4) PCE4(S4) PCE5(S4) 
PCE6(S4) PCE7(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 32 bits
acpihpet0 at acpi0: 14318180 Hz
acpimcfg0 at acpi0
acpimcfg0: addr 0xe000, bus 0-255
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: AMD Phenom(tm) II X3 710 Processor, 2612.29 MHz, 10-04-02
cpu0: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,MWAIT,CX16,POPCNT,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,3DNOW2,3DNOW,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,ITSC
cpu0: 64KB 64b/line 2-way I-cache, 64KB 64b/line 2-way D-cache, 512KB 64b/line 
16-way L2 cache, 6MB 64b/line 48-way L3 cache
cpu0: ITLB 32 4KB entries fully associative, 16 4MB entries fully associative
cpu0: DTLB 48 4KB entries fully associative, 48 4MB entries fully associative
cpu0: AMD erratum 721 detected and fixed
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 200MHz
cpu0: mwait min=64, max=64, IBE
cpu1 at mainbus0: apid 1 (application processor)
cpu1: AMD Phenom(tm) II X3 710 Processor, 2611.92 MHz, 10-04-02
cpu1: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,MWAIT,CX16,POPCNT,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,3DNOW2,3DNOW,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,ITSC
cpu1: 64KB 64b/line 2-way I-cache, 64KB 64b/line 2-way D-cache, 512KB 64b/line 
16-way L2 cache, 6MB 64b/line 48-way L3 cache
cpu1: ITLB 32 4KB entries fully associative, 16 4MB entries fully associative
cpu1: DTLB 48 4KB entries fully associative, 48 4MB entries fully associative
cpu1: AMD erratum 721 detected and fixed
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 2 (application processor)
cpu2: AMD Phenom(tm) II X3 710 Processor, 2611.93 MHz, 10-04-02
cpu2: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,MWAIT,CX16,POPCNT,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,3DNOW2,3DNOW,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,ITSC
cpu2: 64KB 64b/line 2-way I-cache, 64KB 64b/line 2-way D-cache, 512KB 64b/line 
16-way L2 cache, 6MB 64b/line 48-way L3 cache
cpu2: ITLB 32 4KB entries fully associative, 16 4MB entries fully associative
cpu2: DTLB 48 4KB entries fully associative, 48 4MB entries fully associative
cpu2: AMD erratum 721 detected and fixed
cpu2: smt 0, core 2, package 0
ioapic0 at mainbus0: apid 2 pa 0xfec0, version 21, 24 pins, remapped
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 4 (P2P_)
acpiprt2 at acpi0: bus 1 (PCE2)
acpiprt3 at acpi0: bus -1 (PCE3)
acpiprt4 at acpi0: bus 2 (PCE4)
acpiprt5 at acpi0: bus -1 (PCE5)
acpiprt6 at acpi0: bus -1 (PCE6)
acpiprt7 at acpi0: bus -1 (PCE7)
acpiprt8 at acpi0: bus -1 (PCE8)
acpiprt9 at acpi0: bus -1 (PCE9)
acpiprt10 at acpi0: bus 3 (PCEA)
acpiprt11 at acpi0: bus -1 (PCEB)
acpiprt12 at acpi0: bus -1 (PCEC)
acpibtn0 at acpi0: PWRB
acpipci0 at acpi0 PCI0
acpicmos0 at acpi0
"PNP0C14" at acpi0 not configured
acpicpu0 at acpi0: C1(@1 halt!), PSS
acpicpu1 at acpi0: C1(@1 halt!), PSS
acpicpu2 at acpi0: C1(@1 halt!), PSS
cpu0: 2612 MHz: speeds: 2600 1900 1400 800 MHz
pci0 at mainbus0 bus 0
0:0:0: mem address conflict 0xe000/0x2000
pchb0 at pci0 dev 0 function 0 "ATI RD780 HT-PCIE" rev 0x00
ppb0 at pci0 dev 2 function 0 "ATI RD790 PCIE" rev 0x00
pci1 at ppb0 bus 1
radeondrm0 at pci1 dev 0 function 0 "ATI Radeon HD 5450" rev 0x00
drm0 at radeondrm0
radeondrm0: msi
azalia0 at pci1 dev 0 function 1 "ATI Radeon HD 5470 Audio" rev 0x00: msi
azalia0: no supported codecs
ppb1 at pci0 dev 4 function 0 "ATI RD790 PCIE&q

abiword segfaults immediately

[James Cook]

I just installed abiword and also just upgraded to the latest snapshot
(and ran pkg_add -u). abiword immediately crashes when I run it. I have
never tried running abiword on OpenBSD before. Is it broken for others
or just for me?

falsifian angel ~ $ abiword
Abort trap (core dumped) 
falsifian angel ~ $ egdb abiword
GNU gdb (GDB) 7.12.1
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-openbsd6.8".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
.
Find the GDB manual and other documentation resources online at:
.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from abiword...(no debugging symbols found)...done.
(gdb) run
Starting program: /usr/local/bin/abiword 

Program received signal SIGSEGV, Segmentation fault.
0x072405ab041a in EV_UnixMenu::synthesizeMenu(_GtkWidget*, bool) ()
   from /usr/local/lib/libabiword-3.0.so
(gdb) bt
#0  0x072405ab041a in EV_UnixMenu::synthesizeMenu(_GtkWidget*, bool) ()
   from /usr/local/lib/libabiword-3.0.so
#1  0x072405ab23c5 in EV_UnixMenuBar::synthesizeMenuBar() ()
   from /usr/local/lib/libabiword-3.0.so
#2  0x072405a3f368 in XAP_UnixFrameImpl::_createTopLevelWindow() ()
   from /usr/local/lib/libabiword-3.0.so
#3  0x0724058e2476 in AP_UnixFrameImpl::_createWindow() ()
   from /usr/local/lib/libabiword-3.0.so
#4  0x0724058e074a in AP_UnixFrame::initialize(_FrameModes) ()
   from /usr/local/lib/libabiword-3.0.so
#5  0x07240589cbcf in AP_UnixApp::newFrame() ()
   from /usr/local/lib/libabiword-3.0.so
#6  0x0724058fdcfa in AP_App::openCmdLineFiles(AP_Args const*) ()
   from /usr/local/lib/libabiword-3.0.so
#7  0x07240589e319 in AP_UnixApp::main(char const*, int, char**) ()
   from /usr/local/lib/libabiword-3.0.so
#8  0x07218c6f2e61 in ?? ()
#9  0x in ?? ()
(gdb) 

-- 
James

Re: Okay to drop Python 2 support to unbreak scikit-learn with Python 3.8?

[James Cook]

On Sun, Oct 25, 2020 at 11:44:29AM -0400, Daniel Dickman wrote:
> Hi James,
> 
> Not sure if you're still looking into this or not, but I took another stab 
> at it. See attached for new a new port for py-threadpoolctl and an sklearn 
> diff.
> 
> I tried to run the regress tests, but it core dumped about half way in for 
> me... which is still better than the current situation where it bails 
> directly after "import sklearn".
> 
> Does this work any better for you?
> 
> Paul, any thoughts? Can this go in?

Thanks! It works fine with my own use-case, which is running
beancount-import.

However I did notice "make test" also fails for py-threadpoolctl.

Below I've included two pieces of "make test" output...

1. The end of "make test" output for py-threadpoolctl, but with a bunch
   of source code replaced with "...":

tests/test_threadpoolctl.py:417:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/local/lib/python3.8/subprocess.py:489: in run
with Popen(*popenargs, **kwargs) as process:
/usr/local/lib/python3.8/subprocess.py:854: in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

...

if errno_num != 0:
err_msg = os.strerror(errno_num)
>   raise child_exception_type(errno_num, err_msg, err_filename)
E   FileNotFoundError: [Errno 2] No such file or directory: 'python'

/usr/local/lib/python3.8/subprocess.py:1702: FileNotFoundError
 3 failed, 9 passed, 37 skipped in 4.03 seconds 
*** Error 1 in . 
(/home/falsifian/w/OpenBSD/ports/lang/python/python.port.mk:264 'do-test': @cd 
/home/falsifian/w/OpenB
SD/ports/pobj/py-thre...)
*** Error 2 in . 
(/home/falsifian/w/OpenBSD/ports/infrastructure/mk/bsd.port.mk:2954 
'/home/falsifian/w/OpenBSD/ports/p
obj/py-threadpoolctl-2.1.0-python3/.test_done')
*** Error 2 in /home/falsifian/w/OpenBSD/ports/sysutils/py-threadpoolctl 
(/home/falsifian/w/OpenBSD/ports/infrastructur
e/mk/bsd.port.mk:2584 'test')

2. For completeness, the end of the "make test" output for
   py-scikit-learn; it segfaults just as you said:

sklearn/preprocessing/tests/test_label.py .. [ 47%]
 [ 47%]
sklearn/semi_supervised/_label_propagation.py ...[ 47%]
sklearn/semi_supervised/tests/test_label_propagation.py .F.  [ 47%]
sklearn/svm/_classes.py ...  [ 47%]
sklearn/svm/tests/test_bounds.py ... [ 47%]
sklearn/svm/tests/test_sparse.py ... [ 48%]
sklearn/svm/tests/test_svm.py ...Segmentation fault (core 
dumped) 
*** Error 139 in . 
(/home/falsifian/w/OpenBSD/ports/lang/python/python.port.mk:264 'do-test': @cd 
/home/falsifian/w/OpenBSD/ports/pobj/py-sc...)
*** Error 2 in . 
(/home/falsifian/w/OpenBSD/ports/infrastructure/mk/bsd.port.mk:2954 
'/home/falsifian/w/OpenBSD/ports/pobj/py-scikit-learn-0.23.2-python3/.test_done')
*** Error 2 in /home/falsifian/w/OpenBSD/ports/math/py-scikit-learn 
(/home/falsifian/w/OpenBSD/ports/infrastructure/mk/bsd.port.mk:2584 'test')


-- 
James

Re: Okay to drop Python 2 support to unbreak scikit-learn with Python 3.8?

[James Cook]

> (more complete stack trace at end).

I forgot to include it. Here's everything after "test session starts".


= test session starts ==
platform openbsd6 -- Python 3.8.6, pytest-4.4.0, py-1.8.0, pluggy-0.11.0
rootdir: 
/home/falsifian/work/OpenBSD/ports/pobj/py-scikit-learn-0.20.4-python3/scikit-learn-0.20.4,
 inifile: setup.cfg
collected 0 items / 1 errors

 ERRORS 
 ERROR collecting test session _
/usr/local/lib/python3.8/site-packages/_pytest/config/__init__.py:435: in 
_importconftest
return self._conftestpath2mod[conftestpath]
E   KeyError: 
local('/home/falsifian/work/OpenBSD/ports/pobj/py-scikit-learn-0.20.4-python3/scikit-learn-0.20.4/lib.openbsd-6.8-amd64-3.8/sklearn/externals/conftest.py')

During handling of the above exception, another exception occurred:
/usr/local/lib/python3.8/site-packages/_pytest/config/__init__.py:441: in 
_importconftest
mod = conftestpath.pyimport()
/usr/local/lib/python3.8/site-packages/py/_path/local.py:701: in pyimport
__import__(modname)
sklearn/__init__.py:64: in 
from .base import clone
sklearn/base.py:14: in 
from .utils.fixes import signature
sklearn/utils/__init__.py:14: in 
from . import _joblib
sklearn/utils/_joblib.py:22: in 
from ..externals import joblib
sklearn/externals/joblib/__init__.py:119: in 
from .parallel import Parallel
sklearn/externals/joblib/parallel.py:32: in 
from .externals.cloudpickle import dumps, loads
sklearn/externals/joblib/externals/cloudpickle/__init__.py:4: in 
from cloudpickle.cloudpickle import *  # noqa
E   ModuleNotFoundError: No module named 'cloudpickle'

During handling of the above exception, another exception occurred:
/usr/local/lib/python3.8/site-packages/py/_path/common.py:377: in visit
for x in Visitor(fil, rec, ignore, bf, sort).gen(self):
/usr/local/lib/python3.8/site-packages/py/_path/common.py:429: in gen
for p in self.gen(subdir):
/usr/local/lib/python3.8/site-packages/py/_path/common.py:418: in gen
dirs = self.optsort([p for p in entries
/usr/local/lib/python3.8/site-packages/py/_path/common.py:419: in 
if p.check(dir=1) and (rec is None or rec(p))])
/usr/local/lib/python3.8/site-packages/_pytest/main.py:660: in _recurse
ihook = self.gethookproxy(dirpath)
/usr/local/lib/python3.8/site-packages/_pytest/main.py:481: in gethookproxy
my_conftestmodules = pm._getconftestmodules(fspath)
/usr/local/lib/python3.8/site-packages/_pytest/config/__init__.py:419: in 
_getconftestmodules
mod = self._importconftest(conftestpath.realpath())
/usr/local/lib/python3.8/site-packages/_pytest/config/__init__.py:458: in 
_importconftest
raise ConftestImportFailure(conftestpath, sys.exc_info())
E   _pytest.config.ConftestImportFailure: 
(local('/home/falsifian/work/OpenBSD/ports/pobj/py-scikit-learn-0.20.4-python3/scikit-learn-0.20.4/lib.openbsd-6.8-amd64-3.8/sklearn/externals/conftest.py'),
 (, ModuleNotFoundError("No module named 
'cloudpickle'"), ))
!!! Interrupted: 1 errors during collection 
=== 1 error in 1.58 seconds 
*** Error 2 in . 
(/home/falsifian/work/OpenBSD/ports/lang/python/python.port.mk:264 'do-test': 
@cd /home/falsifian/work/OpenBSD/ports/pobj/p...)
*** Error 2 in . 
(/home/falsifian/work/OpenBSD/ports/infrastructure/mk/bsd.port.mk:2954 
'/home/falsifian/work/OpenBSD/ports/pobj/py-scikit-learn-0.20.4-python3/.test_done')
*** Error 2 in /home/falsifian/work/OpenBSD/ports/math/py-scikit-learn 
(/home/falsifian/work/OpenBSD/ports/infrastructure/mk/bsd.port.mk:2584 'test')


-- 
James

Re: Okay to drop Python 2 support to unbreak scikit-learn with Python 3.8?

[James Cook]

On Sun, Oct 18, 2020 at 09:12:23PM -0400, Daniel Dickman wrote:
> On Thu, 15 Oct 2020, James Cook wrote:
> 
> > * Paul prefers the latest version, so I stuck with 0.23.2
> 
> An update would be nice, although seems like fixing the issue by just 
> replacing cloudpickle could be fairly straightforward?
> 
> I've attached a proof of concept which does that (while also updating to 
> 0.20.4).

I tried running make test with your version (after moving things in
files one level up) and end up with
sklearn/externals/joblib/externals/cloudpickle/__init__.py:4: in 
from cloudpickle.cloudpickle import *  # noqa
E   ModuleNotFoundError: No module named 'cloudpickle'
(more complete stack trace at end). I haven't looked too deeply, but
where the old cloudpiclke/__init__.py says "from .cloudpickle import *"
the new one says
from cloudpickle.cloudpickle import *  # noqa
from cloudpickle.cloudpickle_fast import CloudPickler, dumps, dump  # noqa
My Python isn't good enough to know off the top of my head what's going
on, but if I have time later I can dig deeper.

> >  even though it requires new ports.
> 
> what are the new ports? happy to work with you to try to get those in.

I think the only new direct dependency is the threadpoolctl Python
library, but that depends in turn on "flit" and "pytest-cov" which I
don't see in ports. I haven't checked whether those have yet more
dependencies that aren't in ports.
https://github.com/joblib/threadpoolctl/blob/master/dev-requirements.txt
 
> > * I don't know what the WANTLIB line does but copied it from Daniel's
> >   patch in case it's important.
> 
> "make port-lib-depends-check" shows these may be needed.
> 
> see the porting checklist in the porting guide. (item #26)

Thanks; I will take a look if I find more time for this (possibly
tomorrow).

-- 
James

Re: Okay to drop Python 2 support to unbreak scikit-learn with Python 3.8?

[James Cook]

On Thu, Oct 15, 2020 at 01:52:31AM -0400, James Cook wrote:
> On October 15, 2020 12:28:01 a.m. EDT, Daniel Dickman  
> wrote:
> >Does the diff below work for you?
> 
> Probably not; when I ran make test earlier with a similar diff I found
> it needs the Python library threadpoolctl which is not yet on ports.
> However there is an intermediate version that works with 3.8 and doesn't
> need threadpoolctl. I will look at my WIP diff tomorrow when I'm at my
> computer.

Here's my work in progress. I don't know if I'll be able to resume work
before the weekend so feel free to pick it up.

Some notes about the change; lines starting ! might be relevant for the
commit message:

* Paul prefers the latest version, so I stuck with 0.23.2 even though it
  requires new ports.

* It's not finished: sysutils/py-threadpoolctl doesn't exist. Of course
  I haven't updated PLIST.

! The new deps are based on doc/developers/advanced_installation.rst
  from the source. I haven't verified that they work.

! I removed SKLEARN_NO_OPENMP=1 because that was removed in scikit-learn
  commit 9876f748.

* I don't know what the WANTLIB line does but copied it from Daniel's
  patch in case it's important.

! This fixes on incompatibility with Python 3.8 in the included
  cloudpickle source; see
  https://github.com/cloudpipe/cloudpickle/issues/266.
  Before this change, simply importing sklearn in python3.8 led to an
  error:
  >>> import sklearn
  ...
  TypeError: an integer is required (got type bytes)

James

diff --git math/py-scikit-learn/Makefile math/py-scikit-learn/Makefile
index 2e8eaf55e6b..faa066c041b 100644
--- math/py-scikit-learn/Makefile
+++ math/py-scikit-learn/Makefile
@@ -2,10 +2,9 @@
 
 COMMENT =  set of python modules for machine learning and data mining
 
-MODPY_EGG_VERSION =0.20.3
+MODPY_EGG_VERSION =0.23.2
 DISTNAME = scikit-learn-${MODPY_EGG_VERSION}
 PKGNAME =  py-${DISTNAME}
-REVISION = 3
 
 CATEGORIES =   math
 
@@ -16,6 +15,8 @@ MAINTAINER =  Paul Irofti 
 # BSD-3
 PERMIT_PACKAGE =   Yes
 
+WANTLIB += ${COMPILER_LIBCXX} m
+
 MODULES =  lang/python
 
 MODPY_PI = Yes
@@ -26,13 +27,16 @@ MODPY_SETUPTOOLS =  Yes
 FLAVORS =  python3
 FLAVOR =   python3
 
-BUILD_DEPENDS =math/py-numpy${MODPY_FLAVOR}
+BUILD_DEPENDS =lang/cython${MODPY_FLAVOR} \
+   math/py-numpy${MODPY_FLAVOR} \
+   math/py-scipy${MODPY_FLAVOR} \
+   sysutils/py-joblib${MODPY_FLAVOR} \
+   sysutils/py-threadpoolctl${MODPY_FLAVOR}
 RUN_DEPENDS =  math/py-numpy${MODPY_FLAVOR} \
math/py-scipy${MODPY_FLAVOR} \
-   sysutils/py-joblib${MODPY_FLAVOR}
-TEST_DEPENDS = lang/cython${MODPY_FLAVOR}
-
-MAKE_ENV +=SKLEARN_NO_OPENMP=1
+   sysutils/py-joblib${MODPY_FLAVOR} \
+   sysutils/py-threadpoolctl${MODPY_FLAVOR}
+TEST_DEPENDS = math/py-pandas${MODPY_FLAVOR}
 
 pre-test:
cp -r ${WRKSRC}/sklearn/datasets/{data,descr} \
diff --git math/py-scikit-learn/distinfo math/py-scikit-learn/distinfo
index 7ff6978b094..5571636dfef 100644
--- math/py-scikit-learn/distinfo
+++ math/py-scikit-learn/distinfo
@@ -1,2 +1,2 @@
-SHA256 (scikit-learn-0.20.3.tar.gz) = 
xQOAKoHeGLi01A0Gn142N5XuRLFgXzi8EEFgyjv+LEE=
-SIZE (scikit-learn-0.20.3.tar.gz) = 11818490
+SHA256 (scikit-learn-0.23.2.tar.gz) = 
IHZvUV5s1vlUVUOH365wXZPHtUTsDmxqXY4Ab29+9IA=
+SIZE (scikit-learn-0.23.2.tar.gz) = 7155833

Okay to drop Python 2 support to unbreak scikit-learn with Python 3.8?

[James Cook]

Ports currently has scikit-learn 0.20.3, which is incompatible with
Python 3.8. Unfortunately there are no versions that support both
Python 2 and 3.8: the last to support Python 2 is 0.20.0, and the
Python 3.8 issue wasn't fixed until later.

I don't think any other ports depend on scikit-learn (based on running
git grep scikit-learn). Is it okay to drop Python 2 support?

Alternatively I could try to write a patch to fix the problem.

(The Python 3.8 incompatibility is described at
https://github.com/cloudpipe/cloudpickle/issues/266 . It affects
old scikit-learn versions because scikit-learn used to "vendor" joblib,
which in turn vendors cloudpickle. Recent scikit-learn versions take
joblib as an external dependency.)

-- 
James

collision in python-3.8.6 when upgrading 6.7 to snapshot

[James Cook]

I'm reporting a slight hiccup upgrading from 6.7 to snapshot. This might
count as my own fault, but reporting it in case it's something that's
supposed to run smoothly. (I was able to recover; no help needed.)

I upgraded from 6.7 to a snapshot (with sysupgrade -s), then ran
"doas pkg_add -u", and got this:


Collision in python-3.8.6: the following files already exist
/usr/local/bin/2to3 from python-3.8.6
/usr/local/bin/pydoc3 from python-3.8.6
/usr/local/bin/python3 from python-3.8.6
/usr/local/bin/python3-config from python-3.8.6
/usr/local/lib/pkgconfig/python3.pc from python-3.8.6
It seems to be a missing package registration
Repair ? [y/N/a] ?
Ambiguous answer
It seems to be a missing package registration
Repair ? [y/N/a] y
Ustar 
[https://cdn.openbsd.org/pub/OpenBSD/6.8/packages/amd64/python-3.8.6.tgz][lib/libpython3.8.so.0.0]:
 Premature end of archive
Adjusting sha for /usr/local/lib/pkg.0H7b8bHGiO from 
sp85EwWmgTielXKJTS0/ZTfeC2ORQs+SBW7GJ5jxJBM= to 
47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=
Read shared items: ok
pkg_add: Installation of python-3.8.6 failed, partial installation recorded as 
partial-python-3.8.6


The files listed were symlinks to Python 3.7 versions of the files. I
have a vague recollection of running some officially-sanctioned tool to
set up those symlinks, but I might be misremembering.

Is this error expected, or should the upgrade process be smart enough to
sort this out? I have a backup from before the upgrade if anyone wants
more information.

After this, pkg_delete partial-python; pkg_add python3; pkg_add -u
completed the upgrade without trouble (though I haven't tested anything).

-- 
James

Re: port/package build request

[James Cook]

On 2020-09-06 16:53, Eyüp Hakan Duran wrote:
> Dear Madam/Sir,
> 
> I just developed an interest in OpenBSD and started using it on my
> ThinkPad. Thank you so much for developing this OS; I find it very clean
> and relatively easy to use for people who have some basic understanding of
> how it should work. I am far from claiming any expertise though and humbly
> learning a lot.
> 
> I do not possess any significant programming background or skills , and
> this is the main reason for this email/request. I would like to
> respectfully request porting protonmail bridge to OpenBSD. It is open
> source, which is in deb format at the link posted below. I truly wish I had
> the skill set to do this myself but unfortunately I don't. I think
> encrypted email provided by protonmail is very much in line with the
> security and privacy objectives of OpenBSD and this port/package would be a
> great contribution to the project. I am very much aware of the relative
> shortage of worker power and time to dedicate to these kinds of tasks in
> OpenBSD project, but I thought I could take a chance and ask.
> 
> Thank you so much for considering. Kind regards,
> 
> Hakan Duran
> 
> https://protonmail.com/bridge/install

As for as I can tell, people are still waiting for this software to be
ported to any BSD, so I am guessing it would be a lot of work. If you
need it, you are probably stuck using a different OS to run it, for now.
Keep in mind you can run Linux in a VM on OpenBSD
(https://www.openbsd.org/faq/faq16.html).

-- 
James