Re: ld-elf.so.1: Shared object "libssl.so.111" not found, required by "pkg" and others

2023-07-02 Thread Mark Millard 
On Jul 2, 2023, at 03:46, Nuno Teixeira  wrote:

> Yes, I will start building my own ports as well as I do it before.
> 
> I'm using poudriere jail:
> ---
> 14amd64  14.0-CURRENT 1400092 amd64 src=/usr/src
> ---
> That I update each time I update main, but it could also be null mount too.

Null mounting is probably unusual, which is why I mentioned it
as an oddity of my context. I do it because I want to be sure
that ports can build based on an install of the same exact
build that I also, separately install in a bectl BE for booting
--sort of a test. I've a few personal patches that I use in my
FreeBSD builds.

I was not intending to suggest that you use a null mount. Just
be sure that, whatever is used, it avoids having libssl.so.111
and libcrypto.so.111 .

> Thanks,
> 
> Mark Millard  escreveu no dia domingo, 2/07/2023 à(s) 
> 09:15:
> Nuno Teixeira  wrote on
> Date: Sun, 02 Jul 2023 05:22:48 UTC :
> 
> > I'm returning to current and installed from
> > 20230622-b95d2237af40-263748-bootonly.iso
> > 
> > and upgraded to cab2d43b83b (amd64).
> > 
> > Did a magnific delete-old and delete-old-libs and now a lot of packages
> > complain about "ld-elf.so.1: Shared object "libssl.so.111" not found,
> > required by..."
> > 
> > To fix it I rebooted with BE from first instalation since I used
> > beinstall.sh for upgrade.
> > 
> > I know that a lot of things happened in the last days with llvm15->llvm16,
> > openssl3, etc.
> > 
> > My question is when can I do a delete-old{-libs}?
> > I'm thinking building pkgs with a updated current on poudriere and then
> > clean up libs?
> 
> The world in use for building packages from ports (such as in the
> poudriere jail if poudriere is used), should not have libssl.so.111
> or libcrypto.so.111 present.
> 
> I've two files (using amd64 as an example):
> 
> A) ~/origins/amd64-origins.txt
> B) ~/origins/amd64-pkgs.txt
> 
> (A) lists the ports I want to build (but not what those are dependent on).
> (B) lists the packages I want to install (but not what those are dependent 
> on).
> 
> The two have to list flavors in different notations, so having just one file
> ends up not being sufficient. Given that I use poudriere, neither file lists
> ports-mgmt/pkg explicitly.
> 
> To build I use:
> 
> # more ~/build-ports-main-amd64-all_ports.sh 
> #! /bin/sh
> poudriere pkgclean -jmain-amd64 -A
> poudriere bulk -jmain-amd64 -w -f ~/origins/amd64-origins.txt
> 
> (You might not want the -w . Also, I have poudriere null mount my own world
> build's installation for use by the jail. In the current context, no
> libssl.so.111 or libcrypto.so.111 is present in the jail for the bulk run.)

This rebuilds all ports. These system library tracking rebuilds
are uncommon and I use them as a cleanout and start from scratch
point. I tend to accumulate experimental builds and installs that
are not intended for long term use.

Most folks probably try to rebuild only a subset of their ports
that would be sufficient: such typically takes less time.

> To install I use:
> 
> # more ~/pkg-update-amd64-sys.sh
> #! /bin/sh
> pkg update \
> && pkg install -f ports-mgmt/pkg \
> && pkg delete -a \
> && pkg install `cat ~/origins/amd64-pkgs.txt`
> 
> (Note: ports that fail to build need to not be listed in
> ~/origins/amd64-pkgs.txt .)

Ignoring how ports-mgmt/pkg is forced to update first, this
cleans out all the other old packages and then installs all
of them: more of that cleanout and start from scratch
activity.

Most folks probably try to install only a subset of their
ports that would be sufficient.

> I deal with delete-old-libs after the packages are installed.
> I do not have any extra non-port software that would need
> rebuilding.
> 
> (In my context, I do all this activity as root.)

-- 
Nuno Teixeira
FreeBSD Committer (ports)

===
Mark Millard
marklmi at yahoo.com

Re: ld-elf.so.1: Shared object "libssl.so.111" not found, required by "pkg" and others

2023-07-02 Thread Nuno Teixeira 
Hello Tomoaki,

Yes, It was a mistake of mine doing it knowing that there were not pkgs
available yet.

But it was a nice error to test out BE :)

I will take a look at your script and I will start building my own pkgs so
they can be in sync with src.

Thanks,

Tomoaki AOKI  escreveu no dia domingo, 2/07/2023
à(s) 08:57:

> On Sun, 02 Jul 2023 14:41:55 +0900 (JST)
> Yasuhiro Kimura  wrote:
>
> > From: Nuno Teixeira 
> > Subject: ld-elf.so.1: Shared object "libssl.so.111" not found, required
> by "pkg" and others
> > Date: Sun, 2 Jul 2023 06:22:48 +0100
> >
> > > Hello all,
> > >
> > > I'm returning to current and installed from
> 20230622-b95d2237af40-263748-bootonly.iso and upgraded to cab2d43b83b
> (amd64).
> > >
> > > Did a magnific delete-old and delete-old-libs and now a lot of
> packages complain about "ld-elf.so.1: Shared object "libssl.so.111" not
> found,
> > > required by..."
> > >
> > > To fix it I rebooted with BE from first instalation since I used
> beinstall.sh for upgrade.
> > >
> > > I know that a lot of things happened in the last days with
> llvm15->llvm16, openssl3, etc.
> > >
> > > My question is when can I do a delete-old{-libs}?
> > > I'm thinking building pkgs with a updated current on poudriere and
> then clean up libs?
> > >
> > > Thanks,
> >
> > The source of the issue is the migration from OpenSSL 1.1.1 to 3.0.
> >
> > So if you use packages built by yourself (e,g. by using poudriere,
> > portmaster, porupgrade, etc. or simply 'make install'), then you
> > should rebuild and reinstall all packages and then should do
> > `make delete-old-libs`.
> >
> > If you use official binary packages, then you should wait until all
> > packages are built with OpenSSL 3.0.
> >
> > HTH.
> >
> > ---
> > Yasuhiro Kimura
>
> FYI:
> I basically never `make delete-old-libs` blindly.
>
> First, run `make check-old-libs` and record the result.
> Then, create an ad-hoc script to check for affected ports installed and
> generate updating script.
> Then, look into the temporary list generated (or generated script) if
> any port should be actually rebuilt.
> Run the generated script if needed.
>
> Attached is the quick and ugly ad-hoc script I used this time.
> Beware! This generates updating script requiring ports-mgmt/pkg_replace.
> Edit it to use whatever you want.
>
>
> If you're using poudriere[-devel], it should rebuild everything.
> I don't use poudriere on main, as it should force me tooo many full
> rebuilds than on stable/* branches.
>
> If letting poudriere to rebuild everything and configured local repo,
> `pkg upgrade` would do the right thing, maybe.
>
>
> --
> Tomoaki AOKI
>


-- 
Nuno Teixeira
FreeBSD Committer (ports)

Re: ld-elf.so.1: Shared object "libssl.so.111" not found, required by "pkg" and others

2023-07-02 Thread Nuno Teixeira 
Hello Mark,

Yes, I will start building my own ports as well as I do it before.

I'm using poudriere jail:
---
14amd64  14.0-CURRENT 1400092 amd64 src=/usr/src
---
That I update each time I update main, but it could also be null mount too.

Thanks,

Mark Millard  escreveu no dia domingo, 2/07/2023 à(s)
09:15:

> Nuno Teixeira  wrote on
> Date: Sun, 02 Jul 2023 05:22:48 UTC :
>
> > I'm returning to current and installed from
> > 20230622-b95d2237af40-263748-bootonly.iso
> > <
> https://download.freebsd.org/snapshots/amd64/amd64/ISO-IMAGES/14.0/FreeBSD-14.0-CURRENT-amd64-20230622-b95d2237af40-263748-bootonly.iso
> >
> > and upgraded to cab2d43b83b (amd64).
> >
> > Did a magnific delete-old and delete-old-libs and now a lot of packages
> > complain about "ld-elf.so.1: Shared object "libssl.so.111" not found,
> > required by..."
> >
> > To fix it I rebooted with BE from first instalation since I used
> > beinstall.sh for upgrade.
> >
> > I know that a lot of things happened in the last days with
> llvm15->llvm16,
> > openssl3, etc.
> >
> > My question is when can I do a delete-old{-libs}?
> > I'm thinking building pkgs with a updated current on poudriere and then
> > clean up libs?
>
> The world in use for building packages from ports (such as in the
> poudriere jail if poudriere is used), should not have libssl.so.111
> or libcrypto.so.111 present.
>
> I've two files (using amd64 as an example):
>
> A) ~/origins/amd64-origins.txt
> B) ~/origins/amd64-pkgs.txt
>
> (A) lists the ports I want to build (but not what those are dependent on).
> (B) lists the packages I want to install (but not what those are dependent
> on).
>
> The two have to list flavors in different notations, so having just one
> file
> ends up not being sufficient. Given that I use poudriere, neither file
> lists
> ports-mgmt/pkg explicitly.
>
> To build I use:
>
> # more ~/build-ports-main-amd64-all_ports.sh
> #! /bin/sh
> poudriere pkgclean -jmain-amd64 -A
> poudriere bulk -jmain-amd64 -w -f ~/origins/amd64-origins.txt
>
> (You might not want the -w . Also, I have poudriere null mount my own world
> build's installation for use by the jail. In the current context, no
> libssl.so.111 or libcrypto.so.111 is present in the jail for the bulk run.)
>
> To install I use:
>
> # more ~/pkg-update-amd64-sys.sh
> #! /bin/sh
> pkg update \
> && pkg install -f ports-mgmt/pkg \
> && pkg delete -a \
> && pkg install `cat ~/origins/amd64-pkgs.txt`
>
> (Note: ports that fail to build need to not be listed in
> ~/origins/amd64-pkgs.txt .)
>
> I deal with delete-old-libs after the packages are installed.
> I do not have any extra non-port software that would need
> rebuilding.
>
> (In my context, I do all this activity as root.)
>
> ===
> Mark Millard
> marklmi at yahoo.com
>
>

-- 
Nuno Teixeira
FreeBSD Committer (ports)

RE: ld-elf.so.1: Shared object "libssl.so.111" not found, required by "pkg" and others

2023-07-02 Thread Mark Millard 
Nuno Teixeira  wrote on
Date: Sun, 02 Jul 2023 05:22:48 UTC :

> I'm returning to current and installed from
> 20230622-b95d2237af40-263748-bootonly.iso
> 
> and upgraded to cab2d43b83b (amd64).
> 
> Did a magnific delete-old and delete-old-libs and now a lot of packages
> complain about "ld-elf.so.1: Shared object "libssl.so.111" not found,
> required by..."
> 
> To fix it I rebooted with BE from first instalation since I used
> beinstall.sh for upgrade.
> 
> I know that a lot of things happened in the last days with llvm15->llvm16,
> openssl3, etc.
> 
> My question is when can I do a delete-old{-libs}?
> I'm thinking building pkgs with a updated current on poudriere and then
> clean up libs?

The world in use for building packages from ports (such as in the
poudriere jail if poudriere is used), should not have libssl.so.111
or libcrypto.so.111 present.

I've two files (using amd64 as an example):

A) ~/origins/amd64-origins.txt
B) ~/origins/amd64-pkgs.txt

(A) lists the ports I want to build (but not what those are dependent on).
(B) lists the packages I want to install (but not what those are dependent on).

The two have to list flavors in different notations, so having just one file
ends up not being sufficient. Given that I use poudriere, neither file lists
ports-mgmt/pkg explicitly.

To build I use:

# more ~/build-ports-main-amd64-all_ports.sh 
#! /bin/sh
poudriere pkgclean -jmain-amd64 -A
poudriere bulk -jmain-amd64 -w -f ~/origins/amd64-origins.txt

(You might not want the -w . Also, I have poudriere null mount my own world
build's installation for use by the jail. In the current context, no
libssl.so.111 or libcrypto.so.111 is present in the jail for the bulk run.)

To install I use:

# more ~/pkg-update-amd64-sys.sh
#! /bin/sh
pkg update \
&& pkg install -f ports-mgmt/pkg \
&& pkg delete -a \
&& pkg install `cat ~/origins/amd64-pkgs.txt`

(Note: ports that fail to build need to not be listed in
~/origins/amd64-pkgs.txt .)

I deal with delete-old-libs after the packages are installed.
I do not have any extra non-port software that would need
rebuilding.

(In my context, I do all this activity as root.)

===
Mark Millard
marklmi at yahoo.com

Re: ld-elf.so.1: Shared object "libssl.so.111" not found, required by "pkg" and others

2023-07-02 Thread Tomoaki AOKI 
On Sun, 02 Jul 2023 14:41:55 +0900 (JST)
Yasuhiro Kimura  wrote:

> From: Nuno Teixeira 
> Subject: ld-elf.so.1: Shared object "libssl.so.111" not found, required by 
> "pkg" and others
> Date: Sun, 2 Jul 2023 06:22:48 +0100
> 
> > Hello all,
> > 
> > I'm returning to current and installed from 
> > 20230622-b95d2237af40-263748-bootonly.iso and upgraded to cab2d43b83b 
> > (amd64).
> > 
> > Did a magnific delete-old and delete-old-libs and now a lot of packages 
> > complain about "ld-elf.so.1: Shared object "libssl.so.111" not found,
> > required by..."
> > 
> > To fix it I rebooted with BE from first instalation since I used 
> > beinstall.sh for upgrade.
> > 
> > I know that a lot of things happened in the last days with llvm15->llvm16, 
> > openssl3, etc.
> > 
> > My question is when can I do a delete-old{-libs}?
> > I'm thinking building pkgs with a updated current on poudriere and then 
> > clean up libs?
> > 
> > Thanks,
> 
> The source of the issue is the migration from OpenSSL 1.1.1 to 3.0.
> 
> So if you use packages built by yourself (e,g. by using poudriere,
> portmaster, porupgrade, etc. or simply 'make install'), then you
> should rebuild and reinstall all packages and then should do
> `make delete-old-libs`.
> 
> If you use official binary packages, then you should wait until all
> packages are built with OpenSSL 3.0.
> 
> HTH.
> 
> ---
> Yasuhiro Kimura

FYI:
I basically never `make delete-old-libs` blindly.

First, run `make check-old-libs` and record the result.
Then, create an ad-hoc script to check for affected ports installed and
generate updating script.
Then, look into the temporary list generated (or generated script) if
any port should be actually rebuilt.
Run the generated script if needed.

Attached is the quick and ugly ad-hoc script I used this time.
Beware! This generates updating script requiring ports-mgmt/pkg_replace.
Edit it to use whatever you want.


If you're using poudriere[-devel], it should rebuild everything.
I don't use poudriere on main, as it should force me tooo many full
rebuilds than on stable/* branches.

If letting poudriere to rebuild everything and configured local repo,
`pkg upgrade` would do the right thing, maybe.


-- 
Tomoaki AOKI
#!/bin/sh

#Posted Feb.25,2015 to FreeBSD-current ML by Mark Millard
#Modified by Tomoaki AOKI to generate upgrading script for portupgrade.

# Define files to (temporarily) generate.
OLDLIB="base_openssl"
CHECKPREFIX1="/lib/libcrypto"
CHECKFORRE1="${CHECKPREFIX1}[^ ]*\.so\.111"
CHECKPREFIX2="/usr/lib/libssl"
CHECKFORRE2="${CHECKPREFIX2}[^ ]*\.so\.111"
CHECKPREFIX3="/usr/lib32/libcrypto"
CHECKFORRE3="${CHECKPREFIX3}[^ ]*\.so\.111"
CHECKPREFIX4="/usr/lib32/libssl"
CHECKFORRE4="${CHECKPREFIX4}[^ ]*\.so\.111"
DETECTED="/tmp/${OLDLIB}deps0"
NeedUpdate="/tmp/${OLDLIB}deps"
ActualScript="/tmp/rebuild_${OLDLIB}_deps.sh"

# echo CHECKFORRE is ${CHECKFORRE}

# Define pre processing procedure if needed (optional).
PreUpdate="CurDir=\`pwd\`"

# Define port updating program of your choice.
UpdateProgram="pkg_replace -l /usr/ports/${OLDLIB}_deps-\`date \"+%Y%m%d%H%M%S\"\`.log -c -m 'WITH+=NVIDIA,NVIDIA_GL DISABLE_VULNERABILITIES=yes' -v -W -w -b -f"

# Define post processing procedure if needed (optional).
PostUpdate="cd \${CurDir} && portsclean -C"

if [ -f ${DETECTED} ] ; then
	rm ${DETECTED}
fi

if [ -f ${NeedUpdate} ] ; then
	rm ${NeedUpdate}
fi

# find /usr/local/*bin* /usr/local/lib* -type f \
# | xargs ldd -f '%p %A\n' 2>&1 | grep "^/lib/libncurses[^ ]*\.so\.8" | cut -w -f2 \
# | xargs ldd -a | egrep '(^/.*:$| /lib/libncurses[^ ]*\.so\.8 )' \
# | grep -B1 " /lib/libncurses" | grep "^/.*:$" | sed -e's;:$;;' \
# | xargs pkg which -q -o | sort -u

## Unfortunetely, this didn't work if second ldd includes shell variable.
## Always edit it manually as CHECKFORRE.
#
#find /usr/local/*bin* /usr/local/lib* -type f \
#| xargs ldd -f '%p %A\n' 2>&1 | grep "^${CHECKFORRE}" | cut -w -f2 \
#| xargs ldd -a | egrep '(^/.*:$| /lib/libncurses[^ ]*\.so\.8 )' \
#| grep -B1 " ${CHECKPREFIX}" | grep "^/.*:$" | sed -e's;:$;;' \
#| xargs pkg which -q -o | sort -u > ${NeedUpdate}

## Replacing single quotes with double quotes for egrep allowed using shell variable.
find /usr/local/*bin* /usr/local/lib* -type f \
| xargs ldd -f '%p %A\n' 2>&1 | grep "^${CHECKFORRE1}" | cut -w -f2 \
| xargs ldd -a | egrep "(^/.*:$| ${CHECKFORRE1} )" \
| grep -B1 " ${CHECKPREFIX1}" | grep "^/.*:$" | sed -e's;:$;;' \
| xargs pkg which -q -o | sort -u > ${DETECTED}
find /usr/local/*bin* /usr/local/lib* -type f \
| xargs ldd -f '%p %A\n' 2>&1 | grep "^${CHECKFORRE2}" | cut -w -f2 \
| xargs ldd -a | egrep "(^/.*:$| ${CHECKFORRE2} )" \
| grep -B1 " ${CHECKPREFIX2}" | grep "^/.*:$" | sed -e's;:$;;' \
| xargs pkg which -q -o | sort -u >> ${DETECTED}
find /usr/local/*bin* /usr/local/lib* -type f \
| xargs ldd -f '%p %A\n' 2>&1 | grep "^${CHECKFORRE3}" | cut -w -f2 \
| xargs ldd -a | egrep "(^/.*:$| ${CHECKFORRE3} )" \
| grep -B1 " ${CHECKPREFIX3}" | grep "^/.*:$" | sed -e's;:$;

Re: ld-elf.so.1: Shared object "libssl.so.111" not found, required by "pkg" and others

2023-07-01 Thread Yasuhiro Kimura 
From: Nuno Teixeira 
Subject: ld-elf.so.1: Shared object "libssl.so.111" not found, required by 
"pkg" and others
Date: Sun, 2 Jul 2023 06:22:48 +0100

> Hello all,
> 
> I'm returning to current and installed from 
> 20230622-b95d2237af40-263748-bootonly.iso and upgraded to cab2d43b83b (amd64).
> 
> Did a magnific delete-old and delete-old-libs and now a lot of packages 
> complain about "ld-elf.so.1: Shared object "libssl.so.111" not found,
> required by..."
> 
> To fix it I rebooted with BE from first instalation since I used beinstall.sh 
> for upgrade.
> 
> I know that a lot of things happened in the last days with llvm15->llvm16, 
> openssl3, etc.
> 
> My question is when can I do a delete-old{-libs}?
> I'm thinking building pkgs with a updated current on poudriere and then clean 
> up libs?
> 
> Thanks,

The source of the issue is the migration from OpenSSL 1.1.1 to 3.0.

So if you use packages built by yourself (e,g. by using poudriere,
portmaster, porupgrade, etc. or simply 'make install'), then you
should rebuild and reinstall all packages and then should do
`make delete-old-libs`.

If you use official binary packages, then you should wait until all
packages are built with OpenSSL 3.0.

HTH.

---
Yasuhiro Kimura