Re: FreeBSD 13 + CertBot + OpenSSL 3 - status?
Am 2023-10-25 11:57, schrieb DutchDaemon - FreeBSD Forums Administrator: On 25/10/2023 11:12, Ronald Klop wrote: And there are also other implementations of the ACME protocol in the ports tree like security/acmetool. I have no experience with them but they might fit your use case. dehydrate and acmetools are currently on the radar to avoid the next (unavoidable) issue with certbot. As there doesn't seem to be a hard dependency on certbot: security/acme.sh (needs curl and socat, the rest is written in shell, I haven't found another acme tool which is more lightweight in terms of dependencies). Bye, Alexander. -- http://www.Leidinger.net alexan...@leidinger.net: PGP 0x8F31830F9F2772BF http://www.FreeBSD.orgnetch...@freebsd.org : PGP 0x8F31830F9F2772BF signature.asc Description: OpenPGP digital signature
Re: FreeBSD 13 + CertBot + OpenSSL 3 - status?
On 25/10/2023 11:12, Vidar Karlsen wrote:
On Wed, Oct 25, 2023 at 09:22:11AM +0200, Dutch Daemon - FreeBSD Forums
Administrator wrote:
On October 24, 2023 14:54:40 DutchDaemon - FreeBSD Forums Administrator
wrote:
Does anyone in 'port land' know what the current developments are wrt
CertBot (or py-crypto under its hood)?
CertBot is happily compiling against OpenSSL 3 from ports, but when
running 'certbot', the crypto side of it talks to the base system
OpenSSL 1.1.1, hence failing because the OpenSSL 1.1.1 library does not
understand the OpenSSL 3 calls made to it.
From what I understood, this was due to an error/regression in
pkgconf(?) which causes some type of 'path reversal' that causes
py-crypto to ignore the OpenSSL it was compiled against, favoring the
base system library.
I either have to revert a whole lot of servers back to OpenSSL 1.1.1w
from ports in order to renew certificates, or wait for "any movement" in
getting the path reversal addressed/fixed.
So: does anyone know where we're at with this?
Memory jog:
Traceback (most recent call last):
File "/usr/local/bin/certbot", line 33, in
sys.exit(load_entry_point('certbot==2.6.0', 'console_scripts', 'certbot')())
File "/usr/local/bin/certbot", line 25, in importlib_load_entry_point
return next(matches).load()
[...]
File "/usr/local/lib/python3.9/site-packages/cryptography/exceptions.py",
line 9, in
from cryptography.hazmat.bindings._rust import exceptions as rust_exceptions
ImportError:
/usr/local/lib/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so:
Undefined symbol "EVP_default_properties_is_fips_enabled"
What solved this problem for me was to apply the v2 patch from the
pkgconf PR 273961 [1].
The next hurdly you'll probably run into [2] can be solved by running
certbot with the following env variable:
CRYPTOGRAPHY_OPENSSL_NO_LEGACY=1
[1]https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=273961
[2]https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=273656
Hope this helps!
This patch certainly did it for me, hope it gets committed soon (if it
doesn't pose a regression hazard). I did not run into the other problem.
OpenPGP_signature.asc
Description: OpenPGP digital signature
Re: FreeBSD 13 + CertBot + OpenSSL 3 - status?
On 25/10/2023 11:12, Ronald Klop wrote:
Hi,
I see that you are compiling certbot to openssl from ports. Apparently
you are running a not often used configuration of the port.
I'm not so sure about that. OpenSSL in ports is usually quite a lot
ahead of base system OpenSSL, which is why I build everything against
it. I'm sure I'm not the only one.
Did you try reaching out to the maintainer of the port
(pyt...@freebsd.org)?
This bug is pretty well-known, and since it originates in pkgconf (not
Python) and there is a lengthy PR about it (with a tentative patch,
which I was told about just now), I decided to just post here to get
some eyeballs, Successfully ;)
And there are also other implementations of the ACME protocol in the
ports tree like security/acmetool. I have no experience with them but
they might fit your use case.
dehydrate and acmetools are currently on the radar to avoid the next
(unavoidable) issue with certbot.
Sorry I can't help you further for now. Maybe others have more handson
experience with running python with openssl111 from ports.
PS: as you have the name "Dutch" in your email please check out the
upcoming Dutch BSD event in November: https://bsdnl.nl/
I can't, but I'll alert the FreeBSD Forums about it; plenty of Dutchies
on there as well.
*Van:* Dutch Daemon - FreeBSD Forums Administrator
*Datum:* woensdag, 25 oktober 2023 09:22
*Aan:* freebsd-po...@freebsd.org
*Onderwerp:* Re: FreeBSD 13 + CertBot + OpenSSL 3 - status?
On October 24, 2023 14:54:40 DutchDaemon - FreeBSD Forums
Administrator wrote:
Does anyone in 'port land' know what the current developments
are wrt CertBot (or py-crypto under its hood)?
CertBot is happily compiling against OpenSSL 3 from ports, but
when running 'certbot', the crypto side of it talks to the
base system OpenSSL 1.1.1, hence failing because the OpenSSL
1.1.1 library does not understand the OpenSSL 3 calls made to it.
From what I understood, this was due to an error/regression in
pkgconf(?) which causes some type of 'path reversal' that
causes py-crypto to ignore the OpenSSL it was compiled
against, favoring the base system library.
I either have to revert a whole lot of servers back to OpenSSL
1.1.1w from ports in order to renew certificates, or wait for
"any movement" in getting the path reversal addressed/fixed.
So: does anyone know where we're at with this?
Memory jog:
Traceback (most recent call last):
File "/usr/local/bin/certbot", line 33, in
sys.exit(load_entry_point('certbot==2.6.0', 'console_scripts',
'certbot')())
File "/usr/local/bin/certbot", line 25, in importlib_load_entry_point
return next(matches).load()
File "/usr/local/lib/python3.9/importlib/metadata.py", line 86,
in load
module = import_module(match.group('module'))
File "/usr/local/lib/python3.9/importlib/__init__.py", line 127,
in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 986, in
_find_and_load_unlocked
File "", line 680, in _load_unlocked
File "", line 850, in
exec_module
File "", line 228, in
_call_with_frames_removed
File "/usr/local/lib/python3.9/site-packages/certbot/main.py",
line 6, in
from certbot._internal import main as internal_main
File
"/usr/local/lib/python3.9/site-packages/certbot/_internal/main.py",
line 21, in
import josepy as jose
File "/usr/local/lib/python3.9/site-packages/josepy/__init__.py",
line 40, in
from josepy.json_util import (
File
"/usr/local/lib/python3.9/site-packages/josepy/json_util.py", line
14, in
from OpenSSL import crypto
File
"/usr/local/lib/python3.9/site-packages/OpenSSL/__init__.py", line
8, in
from OpenSSL import SSL, crypto
File "/usr/local/lib/python3.9/site-packages/OpenSSL/SSL.py",
line 9, in
from OpenSSL._util import (
File "/usr/local/lib/python3.9/site-packages/OpenSSL/_util.py",
line 6, in
from cryptography.hazmat.bindings.openssl.binding import Binding
File
"/usr/local/lib/python3.9/site-packages/cryptography/hazmat/bindings/openssl/binding.py",
line 15, in
from cryptography.exceptions import InternalError
File
"/usr/local/lib/python3.9/site-packages/cryptography/exceptions.py",
line 9, in
from cryptography.hazmat.bindings._rust import exceptions as
rust_exceptions
ImportError:
/*usr/local/lib/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so*:
Undefined symbol "EVP_default_properties_is_fips_enabled"
OpenPGP_signature.asc
Description: OpenPGP digital signature
Re: FreeBSD 13 + CertBot + OpenSSL 3 - status?
On 25/10/2023 11:12, Vidar Karlsen wrote:
On Wed, Oct 25, 2023 at 09:22:11AM +0200, Dutch Daemon - FreeBSD Forums
Administrator wrote:
On October 24, 2023 14:54:40 DutchDaemon - FreeBSD Forums Administrator
wrote:
Does anyone in 'port land' know what the current developments are wrt
CertBot (or py-crypto under its hood)?
CertBot is happily compiling against OpenSSL 3 from ports, but when
running 'certbot', the crypto side of it talks to the base system
OpenSSL 1.1.1, hence failing because the OpenSSL 1.1.1 library does not
understand the OpenSSL 3 calls made to it.
From what I understood, this was due to an error/regression in
pkgconf(?) which causes some type of 'path reversal' that causes
py-crypto to ignore the OpenSSL it was compiled against, favoring the
base system library.
I either have to revert a whole lot of servers back to OpenSSL 1.1.1w
from ports in order to renew certificates, or wait for "any movement" in
getting the path reversal addressed/fixed.
So: does anyone know where we're at with this?
Memory jog:
Traceback (most recent call last):
File "/usr/local/bin/certbot", line 33, in
sys.exit(load_entry_point('certbot==2.6.0', 'console_scripts', 'certbot')())
File "/usr/local/bin/certbot", line 25, in importlib_load_entry_point
return next(matches).load()
[...]
File "/usr/local/lib/python3.9/site-packages/cryptography/exceptions.py",
line 9, in
from cryptography.hazmat.bindings._rust import exceptions as rust_exceptions
ImportError:
/usr/local/lib/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so:
Undefined symbol "EVP_default_properties_is_fips_enabled"
What solved this problem for me was to apply the v2 patch from the
pkgconf PR 273961 [1].
The next hurdly you'll probably run into [2] can be solved by running
certbot with the following env variable:
CRYPTOGRAPHY_OPENSSL_NO_LEGACY=1
[1]https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=273961
[2]https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=273656
Hope this helps!
Once my current Poudriere run ends I will amend pkgconf with this and
rebuild certbot and related.
Also giving security/dehydrate and possible acmetool a trial run to see
if certbot can be avoided.
This is not the first time I've errored out on Python errors that took
quite some time and effort to chase down and get fixed.
Thanks! That was indeed the PR that put me on the scent of pkgconf, but
I stopped tracking it because of the bickering..
OpenPGP_signature.asc
Description: OpenPGP digital signature
Re: FreeBSD 13 + CertBot + OpenSSL 3 - status?
Hi,
I see that you are compiling certbot to openssl from ports. Apparently you are
running a not often used configuration of the port.
Did you try reaching out to the maintainer of the port (pyt...@freebsd.org)?
And there are also other implementations of the ACME protocol in the ports tree
like security/acmetool. I have no experience with them but they might fit your
use case.
Sorry I can't help you further for now. Maybe others have more handson
experience with running python with openssl111 from ports.
PS: as you have the name "Dutch" in your email please check out the upcoming
Dutch BSD event in November: https://bsdnl.nl/
Regards,
Ronald.
Van: Dutch Daemon - FreeBSD Forums Administrator
Datum: woensdag, 25 oktober 2023 09:22
Aan: freebsd-po...@freebsd.org
Onderwerp: Re: FreeBSD 13 + CertBot + OpenSSL 3 - status?
On October 24, 2023 14:54:40 DutchDaemon - FreeBSD Forums Administrator
wrote:
Does anyone in 'port land' know what the current developments are wrt CertBot (or py-crypto under its hood)?
CertBot is happily compiling against OpenSSL 3 from ports, but when running
'certbot', the crypto side of it talks to the base system OpenSSL 1.1.1, hence
failing because the OpenSSL 1.1.1 library does not understand the OpenSSL 3
calls made to it.
From what I understood, this was due to an error/regression in pkgconf(?) which
causes some type of 'path reversal' that causes py-crypto to ignore the OpenSSL
it was compiled against, favoring the base system library.
I either have to revert a whole lot of servers back to OpenSSL 1.1.1w from ports in order
to renew certificates, or wait for "any movement" in getting the path reversal
addressed/fixed.
So: does anyone know where we're at with this?
Memory jog:
Traceback (most recent call last):
File "/usr/local/bin/certbot", line 33, in
sys.exit(load_entry_point('certbot==2.6.0', 'console_scripts', 'certbot')())
File "/usr/local/bin/certbot", line 25, in importlib_load_entry_point
return next(matches).load()
File "/usr/local/lib/python3.9/importlib/metadata.py", line 86, in load
module = import_module(match.group('module'))
File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in
import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 986, in _find_and_load_unlocked
File "", line 680, in _load_unlocked
File "", line 850, in exec_module
File "", line 228, in _call_with_frames_removed
File "/usr/local/lib/python3.9/site-packages/certbot/main.py", line 6, in
from certbot._internal import main as internal_main
File "/usr/local/lib/python3.9/site-packages/certbot/_internal/main.py", line 21, in
import josepy as jose
File "/usr/local/lib/python3.9/site-packages/josepy/__init__.py", line 40, in
from josepy.json_util import (
File "/usr/local/lib/python3.9/site-packages/josepy/json_util.py", line 14, in
from OpenSSL import crypto
File "/usr/local/lib/python3.9/site-packages/OpenSSL/__init__.py", line 8, in
from OpenSSL import SSL, crypto
File "/usr/local/lib/python3.9/site-packages/OpenSSL/SSL.py", line 9, in
from OpenSSL._util import (
File "/usr/local/lib/python3.9/site-packages/OpenSSL/_util.py", line 6, in
from cryptography.hazmat.bindings.openssl.binding import Binding
File
"/usr/local/lib/python3.9/site-packages/cryptography/hazmat/bindings/openssl/binding.py",
line 15, in
from cryptography.exceptions import InternalError
File "/usr/local/lib/python3.9/site-packages/cryptography/exceptions.py", line 9, in
from cryptography.hazmat.bindings._rust import exceptions as rust_exceptions
ImportError:
/usr/local/lib/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so:
Undefined symbol "EVP_default_properties_is_fips_enabled"
Re: FreeBSD 13 + CertBot + OpenSSL 3 - status?
On Wed, Oct 25, 2023 at 09:22:11AM +0200, Dutch Daemon - FreeBSD Forums
Administrator wrote:
> On October 24, 2023 14:54:40 DutchDaemon - FreeBSD Forums Administrator
> wrote:
> > Does anyone in 'port land' know what the current developments are wrt
> > CertBot (or py-crypto under its hood)?
> > CertBot is happily compiling against OpenSSL 3 from ports, but when
> > running 'certbot', the crypto side of it talks to the base system
> > OpenSSL 1.1.1, hence failing because the OpenSSL 1.1.1 library does not
> > understand the OpenSSL 3 calls made to it.
> > From what I understood, this was due to an error/regression in
> > pkgconf(?) which causes some type of 'path reversal' that causes
> > py-crypto to ignore the OpenSSL it was compiled against, favoring the
> > base system library.
> > I either have to revert a whole lot of servers back to OpenSSL 1.1.1w
> > from ports in order to renew certificates, or wait for "any movement" in
> > getting the path reversal addressed/fixed.
> > So: does anyone know where we're at with this?
>
>
> Memory jog:
>
>
> Traceback (most recent call last):
> File "/usr/local/bin/certbot", line 33, in
> sys.exit(load_entry_point('certbot==2.6.0', 'console_scripts', 'certbot')())
> File "/usr/local/bin/certbot", line 25, in importlib_load_entry_point
> return next(matches).load()
[...]
> File "/usr/local/lib/python3.9/site-packages/cryptography/exceptions.py",
> line 9, in
> from cryptography.hazmat.bindings._rust import exceptions as rust_exceptions
> ImportError:
> /usr/local/lib/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so:
> Undefined symbol "EVP_default_properties_is_fips_enabled"
What solved this problem for me was to apply the v2 patch from the
pkgconf PR 273961 [1].
The next hurdly you'll probably run into [2] can be solved by running
certbot with the following env variable:
CRYPTOGRAPHY_OPENSSL_NO_LEGACY=1
[1] https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=273961
[2] https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=273656
Hope this helps!
--
Vidar
Re: FreeBSD 13 + CertBot + OpenSSL 3 - status?
On October 24, 2023 14:54:40 DutchDaemon - FreeBSD Forums Administrator
wrote:
Does anyone in 'port land' know what the current developments are wrt
CertBot (or py-crypto under its hood)?
CertBot is happily compiling against OpenSSL 3 from ports, but when running
'certbot', the crypto side of it talks to the base system OpenSSL 1.1.1,
hence failing because the OpenSSL 1.1.1 library does not understand the
OpenSSL 3 calls made to it.
From what I understood, this was due to an error/regression in pkgconf(?)
which causes some type of 'path reversal' that causes py-crypto to ignore
the OpenSSL it was compiled against, favoring the base system library.
I either have to revert a whole lot of servers back to OpenSSL 1.1.1w from
ports in order to renew certificates, or wait for "any movement" in getting
the path reversal addressed/fixed.
So: does anyone know where we're at with this?
Memory jog:
Traceback (most recent call last):
File "/usr/local/bin/certbot", line 33, in
sys.exit(load_entry_point('certbot==2.6.0', 'console_scripts', 'certbot')())
File "/usr/local/bin/certbot", line 25, in importlib_load_entry_point
return next(matches).load()
File "/usr/local/lib/python3.9/importlib/metadata.py", line 86, in load
module = import_module(match.group('module'))
File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in
import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 986, in _find_and_load_unlocked
File "", line 680, in _load_unlocked
File "", line 850, in exec_module
File "", line 228, in _call_with_frames_removed
File "/usr/local/lib/python3.9/site-packages/certbot/main.py", line 6, in
from certbot._internal import main as internal_main
File "/usr/local/lib/python3.9/site-packages/certbot/_internal/main.py",
line 21, in
import josepy as jose
File "/usr/local/lib/python3.9/site-packages/josepy/__init__.py", line 40,
in
from josepy.json_util import (
File "/usr/local/lib/python3.9/site-packages/josepy/json_util.py", line
14, in
from OpenSSL import crypto
File "/usr/local/lib/python3.9/site-packages/OpenSSL/__init__.py", line 8,
in
from OpenSSL import SSL, crypto
File "/usr/local/lib/python3.9/site-packages/OpenSSL/SSL.py", line 9, in
from OpenSSL._util import (
File "/usr/local/lib/python3.9/site-packages/OpenSSL/_util.py", line 6, in
from cryptography.hazmat.bindings.openssl.binding import Binding
File
"/usr/local/lib/python3.9/site-packages/cryptography/hazmat/bindings/openssl/binding.py",
line 15, in
from cryptography.exceptions import InternalError
File "/usr/local/lib/python3.9/site-packages/cryptography/exceptions.py",
line 9, in
from cryptography.hazmat.bindings._rust import exceptions as rust_exceptions
ImportError:
/usr/local/lib/python3.9/site-packages/cryptography/hazmat/bindings/_rust.abi3.so:
Undefined symbol "EVP_default_properties_is_fips_enabled"
Re: FreeBSD 13 + CertBot + OpenSSL 3 - status?
On 24/10/2023 18:24, Matthew Seaman wrote: On 24/10/2023 13:54, DutchDaemon - FreeBSD Forums Administrator wrote: Does anyone in 'port land' know what the current developments are wrt CertBot (or py-crypto under its hood)? CertBot is happily compiling against OpenSSL 3 from ports, but when running 'certbot', the crypto side of it talks to the base system OpenSSL 1.1.1, hence failing because the OpenSSL 1.1.1 library does not understand the OpenSSL 3 calls made to it. From what I understood, this was due to an error/regression in pkgconf(?) which causes some type of 'path reversal' that causes py-crypto to ignore the OpenSSL it was compiled against, favoring the base system library. I either have to revert a whole lot of servers back to OpenSSL 1.1.1w from ports in order to renew certificates, or wait for "any movement" in getting the path reversal addressed/fixed. So: does anyone know where we're at with this? certbot is running just fine for me on stable/14 with openssl 3.x from ports. Note that stable/14 has openssl 3.x in base. Cheers, Matthew Yes ;) I knew that that would be 'the other option', but tracking -RELEASE and its patch levels is currently preferred over here. Got a tip about 'dehydrated', so maybe that'll work for now, until 14-REL is on the books. OpenPGP_signature.asc Description: OpenPGP digital signature
Re: FreeBSD 13 + CertBot + OpenSSL 3 - status?
On 24/10/2023 13:54, DutchDaemon - FreeBSD Forums Administrator wrote: Does anyone in 'port land' know what the current developments are wrt CertBot (or py-crypto under its hood)? CertBot is happily compiling against OpenSSL 3 from ports, but when running 'certbot', the crypto side of it talks to the base system OpenSSL 1.1.1, hence failing because the OpenSSL 1.1.1 library does not understand the OpenSSL 3 calls made to it. From what I understood, this was due to an error/regression in pkgconf(?) which causes some type of 'path reversal' that causes py-crypto to ignore the OpenSSL it was compiled against, favoring the base system library. I either have to revert a whole lot of servers back to OpenSSL 1.1.1w from ports in order to renew certificates, or wait for "any movement" in getting the path reversal addressed/fixed. So: does anyone know where we're at with this? certbot is running just fine for me on stable/14 with openssl 3.x from ports. Note that stable/14 has openssl 3.x in base. Cheers, Matthew OpenPGP_signature.asc Description: OpenPGP digital signature
