SOLVED Re: any way to "redo" a botched upgrade?

[Jonathan Thornburg]

Thank you to all who responded in this thread and by private email --
your replies were very helpful!  Following Stuart Henderson's suggestion,
I found that I did indeed have a
  /usr/local/libdata/perl5/site_perl/amd64-openbsd/auto/Term/ReadLine/Gnu
containing (among other things) an out-of-date file Gnu.so.

I have now "solved" the problem by replacing the entire
  /usr/local/libdata/perl5/
tree with a copy of that from a twin machine (also amd64/7.3, just
upgraded from 7.2) which doesn't seem to have this problem.

After that the immediate problem (fatal error on /use Term::ReadPassword;/)
is gone.  I did another full 'pkg_add -uvv' to be on the safe side, which
found a few 'file already exists', but after overwriting those everything
seems to be working now.

Thanks again to everyone who helped!
ciao,
-- 
-- "Jonathan Thornburg [remove -color to reply]" 
   on the west coast of Canada
   "Now back when I worked in banking, if someone went to Barclays,
pretended to be me, borrowed UKP10,000 and legged it, that was
`impersonation', and it was the bank's money that had been stolen,
not my identity.  How did things change?" -- Ross Anderson

Re: any way to "redo" a botched upgrade?

[Andrew Hewus Fresh]

On Thu, Apr 20, 2023 at 12:49:35AM -0700, Jonathan Thornburg wrote:
> Andrew Hewus Fresh  wrote:
> > This usually happens when an XS module is installed outside of the
> > package ecosystem, often with a CPAN client.
> > 
> > I would guess this error is Term::ReadLine::Gnu
> > https://metacpan.org/pod/Term::ReadLine::Gnu
> 
> I've never used the CPAN client (I always install CPAN perl modules
> "manually" via 'perl Makefile.PL'; 'make'; 'make test'; 'doas make install',
> and follow dependency chains manually).

This also would have the same effect.  The cpan tooling is just
wrappers to do just this for you.

I just guessed at Term::ReadLine::Gnu, you could search for the Gnu.so
in /usr/local/libexec/perl5 and see what the actual module is.


l8rZ,
-- 
andrew

Hey! It compiles! Ship it!

Re: any way to "redo" a botched upgrade?

[Stuart Henderson]

On 2023-04-20, Jonathan Thornburg  wrote:
>
> Andrew Hewus Fresh  wrote:
>> This usually happens when an XS module is installed outside of the
>> package ecosystem, often with a CPAN client.
>> 
>> I would guess this error is Term::ReadLine::Gnu
>> https://metacpan.org/pod/Term::ReadLine::Gnu
>
> I've never used the CPAN client (I always install CPAN perl modules
> "manually" via 'perl Makefile.PL'; 'make'; 'make test'; 'doas make install',
> and follow dependency chains manually).  And I've never installed
> Term::Readline::Gnu.
> 
> But I do see that I installed the p5-Devel-NYTProf
> package, and maybe that dragged in some conflicting library?  But
> deleting that package doesn't change the error I'm getting.

p5-Term-ReadLine-Gnu is in ports, but NYTProf doesn't depend on it.
There are a few other ports which do though.

Take a look and see if you have

/usr/local/libdata/perl5/site_perl/amd64-openbsd/auto/Term/ReadLine/Gnu 

It is possible that it was installed from packages before, but
you lost the files for it in /var/db/pkg (can happen after a crash
or power loss at a bad time) which would prevent it from being
updated by pkg_add. If present, timestamps on that file might
give clues.

Alternatively if you run your test script with LD_DEBUG set in the
environment, you should have some "dlopen: loading" lines in the
output, one of which will show the path where it found this module.
There will be a bunch of output, you'll need scrollback or script(1)
or something.

Re: any way to "redo" a botched upgrade?

[Jonathan Thornburg]

In message 
I described an amd64 system which, after a 7.1-->7.2-->7.3 set of
upgrades, now has the perl Term::ReadPassword library inconsistent with
the system perl.


Jeremy Mates  asked
> Does the code error in the same way after the Term::ReadPassword package
> is removed, thus indicating that maybe a `cpan` run installed the module
> outside the OpenBSD package system?

Good idea, but no.  If the package is removed I get the expected failure
that perl can't find the package:
  # pkg_delete -vv !$
  pkg_delete -vv p5-Term-ReadPassword
  Running /usr/sbin/makewhatis -u /usr/local/man -- 
/usr/local/man/man3p/Term::ReadPassword.3p
  p5-Term-ReadPassword-0.11p2: ok
  Read shared items: ok
  Clean shared items: ok
  /dev/sd1g on /usr/local: -18737 bytes
  # /tmp/foo
  Can't locate Term/ReadPassword.pm in @INC (you may need to install the 
Term::ReadPassword module) (@INC contains: 
/usr/local/libdata/perl5/site_perl/amd64-openbsd 
/usr/local/libdata/perl5/site_perl /usr/libdata/perl5/amd64-openbsd 
/usr/libdata/perl5) at /tmp/foo line 4.
  BEGIN failed--compilation aborted at /tmp/foo line 4.
  #


Andrew Hewus Fresh  wrote:
> This usually happens when an XS module is installed outside of the
> package ecosystem, often with a CPAN client.
> 
> I would guess this error is Term::ReadLine::Gnu
> https://metacpan.org/pod/Term::ReadLine::Gnu

I've never used the CPAN client (I always install CPAN perl modules
"manually" via 'perl Makefile.PL'; 'make'; 'make test'; 'doas make install',
and follow dependency chains manually).  And I've never installed
Term::Readline::Gnu.  But I do see that I installed the p5-Devel-NYTProf
package, and maybe that dragged in some conflicting library?  But
deleting that package doesn't change the error I'm getting.

Hmm, since I have another -- fully working 7.3/amd64 system with what
should be the identical set of perl modules and packages, maybe comparing
recursive directory listings and/or file contents of the perl @INC
directories between the working and broken systems would be informative.
I'll try that.

Thanks to both of you for your suggestions!
ciao,
--
-- "Jonathan Thornburg [remove -color to reply]" 
   on the west coast of Canada, eh?
   "Now back when I worked in banking, if someone went to Barclays,
pretended to be me, borrowed UKP10,000 and legged it, that was
`impersonation', and it was the bank's money that had been stolen,
not my identity.  How did things change?" -- Ross Anderson

Re: any way to "redo" a botched upgrade?

[Andrew Hewus Fresh]

On Wed, Apr 19, 2023 at 07:25:17PM -0700, Jonathan Thornburg wrote:
> I've just upgraded an amd64 machine from 7.1 to 7.3 (first a 7.1-->7.2
> upgrade, immediately followed by a 7.2-->7.3 upgrade, both following the
> FAQ instructions).  After a full 'pkg_add -uvv', at least one package
> (p5-Term-ReadPassword) is out-of-sync with the new perl binary:
>   # cat /tmp/foo
>   #!/usr/bin/perl
>   use warnings;
>   use strict;
>   use Term::ReadPassword;
>   
>   print "hello, world\n";
>   # /tmp/foo
>   Gnu.c: loadable library and perl binaries are mismatched (got first 
> handshake key 0xec0, needed 0xeb8)

This usually happens when an XS module is installed outside of the
package ecosystem, often with a CPAN client.

I would guess this error is Term::ReadLine::Gnu
https://metacpan.org/pod/Term::ReadLine::Gnu

You might be able to pkg_add p5-Term-ReadLine-Gnu to overwrite that
file, but not sure that will work.

I recommend looking at sysutils/sysclean as the easiest way to clean up
those files.

l8rZ,
-- 
andrew

Adding manpower to a late software project makes it later.

any way to "redo" a botched upgrade?

[Jonathan Thornburg]

I've just upgraded an amd64 machine from 7.1 to 7.3 (first a 7.1-->7.2
upgrade, immediately followed by a 7.2-->7.3 upgrade, both following the
FAQ instructions).  After a full 'pkg_add -uvv', at least one package
(p5-Term-ReadPassword) is out-of-sync with the new perl binary:
  # cat /tmp/foo
  #!/usr/bin/perl
  use warnings;
  use strict;
  use Term::ReadPassword;
  
  print "hello, world\n";
  # /tmp/foo
  Gnu.c: loadable library and perl binaries are mismatched (got first handshake 
key 0xec0, needed 0xeb8)
  # pkg_add -uvv p5-Term-ReadPassword
  Update candidates: quirks-6.121 -> quirks-6.121
  quirks-6.121 signed on 2023-04-19T08:30:26Z
  No change in quirks-6.121
  Update candidates: p5-Term-ReadPassword-0.11p2 -> p5-Term-ReadPassword-0.11p2
  No change in p5-Term-ReadPassword-0.11p2
  #

I've tried deleting and re-adding the p5-Term-ReadPassword package
('pkg_delete -vv p5-Term-ReadPassword', 'pkg_add -vv p5-Term-ReadPassword')
and rebooting, but this didn't change the above behavior.  My /etc/installurl
points to 
  https://cdn.openbsd.org/pub/OpenBSD

The output of 'perl -V' on this system is identical to that on another
amd64 machine (which I just upgraded from 7.2-->7.3) which does *not*
have this problem.  In both cases:
# perl -V
Summary of my perl5 (revision 5 version 36 subversion 0) configuration:
   
  Platform:
osname=openbsd
osvers=7.3
archname=amd64-openbsd
uname='openbsd'
config_args='-dse -Dopenbsd_distribution=defined -Dmksymlinks'
hint=recommended
useposix=true
d_sigaction=define
useithreads=undef
usemultiplicity=undef
use64bitint=define
use64bitall=define
uselongdouble=undef
usemymalloc=n
default_inc_excludes_dot=define
  Compiler:
cc='cc'
ccflags ='-DNO_LOCALE_NUMERIC -DNO_LOCALE_COLLATE -fno-strict-aliasing 
-fno-delete-null-pointer-checks -pipe -fstack-protector-strong 
-I/usr/local/include'
optimize='-O2'
cppflags='-DBIG_TIME -DNO_LOCALE_NUMERIC -DNO_LOCALE_COLLATE 
-fno-strict-aliasing -fno-delete-null-pointer-checks -pipe 
-fstack-protector-strong -I/usr/local/include'
ccversion=''
gccversion='OpenBSD Clang 13.0.0'
gccosandvers=''
intsize=4
longsize=8
ptrsize=8
doublesize=8
byteorder=12345678
doublekind=3
d_longlong=define
longlongsize=8
d_longdbl=define
longdblsize=16
longdblkind=3
ivtype='long'
ivsize=8
nvtype='double'
nvsize=8
Off_t='off_t'
lseeksize=8
alignbytes=8
prototype=define
  Linker and Libraries:
ld='cc'
ldflags ='-Wl,-E  -fstack-protector-strong -L/usr/local/lib'
libpth=/usr/lib /usr/lib/clang/13.0.0/lib
libs=-lm -lc
perllibs=-lm -lc
libc=/usr/lib/libc.so.97.0
so=so
useshrplib=true
libperl=libperl.so.23.0
gnulibc_version=''
  Dynamic Linking:
dlsrc=dl_dlopen.xs
dlext=so
d_dlsymun=undef
ccdlflags='-Wl,-R/usr/libdata/perl5/amd64-openbsd/CORE'
cccdlflags='-DPIC -fpic '
lddlflags='-shared -fpic  -fstack-protector-strong -L/usr/local/lib'


Characteristics of this binary (from libperl): 
  Compile-time options:
HAS_TIMES
PERLIO_LAYERS
PERL_COPY_ON_WRITE
PERL_DONT_CREATE_GVSV
PERL_MALLOC_WRAP
PERL_OP_PARENT
PERL_PRESERVE_IVUV
USE_64_BIT_ALL
USE_64_BIT_INT
USE_LARGE_FILES
USE_LOCALE
USE_LOCALE_CTYPE
USE_LOCALE_TIME
USE_PERLIO
USE_PERL_ATOF
  Built under openbsd
  @INC:
/usr/local/libdata/perl5/site_perl/amd64-openbsd
/usr/local/libdata/perl5/site_perl
/usr/libdata/perl5/amd64-openbsd
/usr/libdata/perl5
#

I presume that I somehow botched one of the upgrades.
Is there any easy way to "redo the upgrade", or should I just give
up and do a clean 7.3 (re)install (followed by manual re-creation of
all of my system configuration)?

Thanks,
-- 
-- "Jonathan Thornburg [remove -color to reply]" 
   on the west coast of Canada, eh?
   "Now back when I worked in banking, if someone went to Barclays,
pretended to be me, borrowed UKP10,000 and legged it, that was
`impersonation', and it was the bank's money that had been stolen,
not my identity.  How did things change?" -- Ross Anderson