Bug#926315: [Pkg-openssl-devel] Bug#926315: openssl: wget https://google.com fails in d-i

2019-04-03 Thread Kurt Roeckx 
On Wed, Apr 03, 2019 at 10:03:13PM +0200, Sebastian Andrzej Siewior wrote:
> On 2019-04-03 11:14:54 [+0100], Dimitri John Ledkov wrote:
> > $ wget https://google.com
> > 
> > fails in Buster alpha installer, when used from a booted netinst iso
> > in a tty. It also means that fetch-url fails, and thus one cannot use
> > https preseeding.
> > 
> > A fix/workaround, is $ touch /usr/lib/ssl/openssl.cnf it appears that
> > openssl requires for that file to be present, and it cannot be a
> > dangling symlink. However, in udeb environment such file does not
> > exists. I guess that maybe libssl1.1-udeb should ship an empty
> > openssl.cnf there, or ship the regular deb's /etc/ssl/openssl.cnf in
> > /usr/lib/ssl/openssl.cnf in the udeb.
> 
> interresting.
> Kurt: should we provide the openssl.cnf and move it from openssl to
> libssl1.1 as well or should we rather treat the missing openssl.cnf as
> okay?

I think shipping it in the libssl1.1 .deb is going to complicate
upgrades, so I rather not do that. I don't see a problem doing it
in the .udeb.

I'm not sure why not having the config file causes problems. I
think it should be possible to run without config file, so I would
at least like to know first why it fails.


Kurt

Bug#926315: [Pkg-openssl-devel] Bug#926315: openssl: wget https://google.com fails in d-i

2019-04-03 Thread Cyril Brulebois 
Hi,

Thanks for looping me in, extending to debian-boot@.

Kurt Roeckx  (2019-04-03):
> On Wed, Apr 03, 2019 at 10:03:13PM +0200, Sebastian Andrzej Siewior wrote:
> > On 2019-04-03 11:14:54 [+0100], Dimitri John Ledkov wrote:
> > > $ wget https://google.com
> > > 
> > > fails in Buster alpha installer, when used from a booted netinst iso
> > > in a tty. It also means that fetch-url fails, and thus one cannot use
> > > https preseeding.
> > > 
> > > A fix/workaround, is $ touch /usr/lib/ssl/openssl.cnf it appears that
> > > openssl requires for that file to be present, and it cannot be a
> > > dangling symlink. However, in udeb environment such file does not
> > > exists. I guess that maybe libssl1.1-udeb should ship an empty
> > > openssl.cnf there, or ship the regular deb's /etc/ssl/openssl.cnf in
> > > /usr/lib/ssl/openssl.cnf in the udeb.
> > 
> > interresting.
> > Kurt: should we provide the openssl.cnf and move it from openssl to
> > libssl1.1 as well or should we rather treat the missing openssl.cnf as
> > okay?
> 
> I think shipping it in the libssl1.1 .deb is going to complicate
> upgrades, so I rather not do that. I don't see a problem doing it
> in the .udeb.
> 
> I'm not sure why not having the config file causes problems. I
> think it should be possible to run without config file, so I would
> at least like to know first why it fails.

I'm pretty sure we had successes with wget/https within d-i not so long
ago (i.e. during the last BSP at Mozilla's, past week), and there were
no changes on the openssl side in the meanwhile.

I'll be double checking using the month worth of dailies we have, and
report back with my findings.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#926315: [Pkg-openssl-devel] Bug#926315: openssl: wget https://google.com fails in d-i

2019-04-03 Thread Cyril Brulebois 
Hi again,

Cyril Brulebois  (2019-04-03):
> I'm pretty sure we had successes with wget/https within d-i not so long
> ago (i.e. during the last BSP at Mozilla's, past week), and there were no
> changes on the openssl side in the meanwhile.

Or maybe my explicit testing didn't happen at the right time.

> I'll be double checking using the month worth of dailies we have, and
> report back with my findings.

Anyway: downgrading libssl1.1-udeb to the previous version (1.1.1a-1) isn't
sufficient; I've had to downgrade its companion library, libcrypto1.1-udeb.

I've included strace-udeb in the image, and extracted a trace. It seems the
mere existence (or lack thereof) of the configuration file is what triggers
the early exit. Running “touch” on it is sufficient to get a successful
connection/download, with the current version of both libraries (1.1.1b-1).


1726  execve("/usr/bin/wget", ["wget", "https://google.fr";], ["USER=root", 
"HOME=/", "TERM=linux", "BOOT_IMAGE=linux", 
"PATH=/sbin:/usr/sbin:/bin:/usr/bin", "vga=788", "SHELL=/bin/sh", 
"initrd=initrd.gz", "PWD=/"]) = 0
1726  brk(NULL) = 0x56007142f000
1726  access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or 
directory)
1726  openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = -1 ENOENT 
(No such file or directory)
1726  openat(AT_FDCWD, 
"/lib/x86_64-linux-gnu/tls/x86_64/x86_64/libpcre2-8.so.0", O_RDONLY|O_CLOEXEC) 
= -1 ENOENT (No such file or directory)
1726  stat("/lib/x86_64-linux-gnu/tls/x86_64/x86_64", 0x7ffd7454c960) = -1 
ENOENT (No such file or directory)
1726  openat(AT_FDCWD, "/lib/x86_64-linux-gnu/tls/x86_64/libpcre2-8.so.0", 
O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1726  stat("/lib/x86_64-linux-gnu/tls/x86_64", 0x7ffd7454c960) = -1 ENOENT 
(No such file or directory)
1726  openat(AT_FDCWD, "/lib/x86_64-linux-gnu/tls/x86_64/libpcre2-8.so.0", 
O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1726  stat("/lib/x86_64-linux-gnu/tls/x86_64", 0x7ffd7454c960) = -1 ENOENT 
(No such file or directory)
1726  openat(AT_FDCWD, "/lib/x86_64-linux-gnu/tls/libpcre2-8.so.0", 
O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1726  stat("/lib/x86_64-linux-gnu/tls", 0x7ffd7454c960) = -1 ENOENT (No 
such file or directory)
1726  openat(AT_FDCWD, 
"/lib/x86_64-linux-gnu/x86_64/x86_64/libpcre2-8.so.0", O_RDONLY|O_CLOEXEC) = -1 
ENOENT (No such file or directory)
1726  stat("/lib/x86_64-linux-gnu/x86_64/x86_64", 0x7ffd7454c960) = -1 
ENOENT (No such file or directory)
1726  openat(AT_FDCWD, "/lib/x86_64-linux-gnu/x86_64/libpcre2-8.so.0", 
O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1726  stat("/lib/x86_64-linux-gnu/x86_64", 0x7ffd7454c960) = -1 ENOENT (No 
such file or directory)
1726  openat(AT_FDCWD, "/lib/x86_64-linux-gnu/x86_64/libpcre2-8.so.0", 
O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1726  stat("/lib/x86_64-linux-gnu/x86_64", 0x7ffd7454c960) = -1 ENOENT (No 
such file or directory)
1726  openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libpcre2-8.so.0", 
O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1726  stat("/lib/x86_64-linux-gnu", {st_dev=makedev(0, 0x2), st_ino=9367, 
st_mode=S_IFDIR|0755, st_nlink=2, st_uid=0, st_gid=0, st_blksize=4096, 
st_blocks=0, st_size=160, st_atime=1554325535 /* 2019-04-03T21:05:35+ */, 
st_atime_nsec=0, st_mtime=1554325535 /* 2019-04-03T21:05:35+ */, 
st_mtime_nsec=0, st_ctime=1554325844 /* 2019-04-03T21:10:44.514891234+ */, 
st_ctime_nsec=514891234}) = 0
1726  openat(AT_FDCWD, 
"/usr/lib/x86_64-linux-gnu/tls/x86_64/x86_64/libpcre2-8.so.0", 
O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1726  stat("/usr/lib/x86_64-linux-gnu/tls/x86_64/x86_64", 0x7ffd7454c960) = 
-1 ENOENT (No such file or directory)
1726  openat(AT_FDCWD, 
"/usr/lib/x86_64-linux-gnu/tls/x86_64/libpcre2-8.so.0", O_RDONLY|O_CLOEXEC) = 
-1 ENOENT (No such file or directory)
1726  stat("/usr/lib/x86_64-linux-gnu/tls/x86_64", 0x7ffd7454c960) = -1 
ENOENT (No such file or directory)
1726  openat(AT_FDCWD, 
"/usr/lib/x86_64-linux-gnu/tls/x86_64/libpcre2-8.so.0", O_RDONLY|O_CLOEXEC) = 
-1 ENOENT (No such file or directory)
1726  stat("/usr/lib/x86_64-linux-gnu/tls/x86_64", 0x7ffd7454c960) = -1 
ENOENT (No such file or directory)
1726  openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/tls/libpcre2-8.so.0", 
O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1726  stat("/usr/lib/x86_64-linux-gnu/tls", 0x7ffd7454c960) = -1 ENOENT (No 
such file or directory)
1726  openat(AT_FDCWD, 
"/usr/lib/x86_64-linux-gnu/x86_64/x86_64/libpcre2-8.so.0", O_RDONLY|O_CLOEXEC) 
= -1 ENOENT (No such file or directory)
1726  stat("/usr/lib/x86_64-linux-gnu/x86_64/x86_64", 0x7ffd7454c960) = -1 
ENOENT (No such file or directory)
1726  openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/x86_64/libpcre2-8.so.0", 
O_RDONLY|O_CLOEX

Bug#926315: [Pkg-openssl-devel] Bug#926315: openssl: wget https://google.com fails in d-i

2019-04-03 Thread Kurt Roeckx 
On Wed, Apr 03, 2019 at 11:23:19PM +0200, Cyril Brulebois wrote:
> 1726  write(2, "Disabling SSL due to encountered errors.\n", 41) = 41

Looking at the source, about the only reason I can see to get that
is that SSL_CTX_new() failed.

If I understand correctly, it's actually a change in libcrypto
between 1.1.1a and 1.1.1b?

The most likely commit to change behaviour seems to be:
commit 25eb9299cec4404a4cdf3167056bd147af2582f3
Author: Viktor Dukhovni 
Date:   Tue Jan 1 02:53:24 2019 -0500

More configurable crypto and ssl library initialization

1.  In addition to overriding the default application name,
one can now also override the configuration file name
and flags passed to CONF_modules_load_file().

2.  By default we still keep going when configuration file
processing fails.  But, applications that want to be
strict about initialization errors can now make explicit
flag choices via non-null OPENSSL_INIT_SETTINGS that omit
the CONF_MFLAGS_IGNORE_RETURN_CODES flag (which had so far
been both undocumented and unused).

3.  In OPENSSL_init_ssl() do not request OPENSSL_INIT_LOAD_CONFIG
if the options already include OPENSSL_INIT_NO_LOAD_CONFIG.

4.  Don't set up atexit() handlers when called with opts equal to
OPENSSL_INIT_BASE_ONLY (this flag should only be used alone).

Reviewed-by: Bernd Edlinger 
Reviewed-by: Matt Caswell 
(Merged from https://github.com/openssl/openssl/pull/7969)

But the commit message at least indicates that it should just continue.

wget in buster actually seems to be linked to gnutls, and trying
other applications just seem to work without config file.


Kurt

Bug#926315: [Pkg-openssl-devel] Bug#926315: openssl: wget https://google.com fails in d-i

2019-04-05 Thread Dimitri John Ledkov 
On Wed, 3 Apr 2019 at 22:57, Kurt Roeckx  wrote:
>
> On Wed, Apr 03, 2019 at 11:23:19PM +0200, Cyril Brulebois wrote:
> > 1726  write(2, "Disabling SSL due to encountered errors.\n", 41) = 41
>
> Looking at the source, about the only reason I can see to get that
> is that SSL_CTX_new() failed.
>
> But the commit message at least indicates that it should just continue.
>
> wget in buster actually seems to be linked to gnutls, and trying
> other applications just seem to work without config file.
>

Using the CTX api is optional, so i expect other apps would fail too
if one forces them to use CTX apis (e.g. like client cert auth) but
it's unlikely to be done in d-i / udeb.

I do think cherrypicking the patch kurt identified should be done.

But I also think that openssl.cnf should be shipped in libssl1.1-udeb
(either in /usr directly - see my patch, or symlink in /usr and a real
file in /etc like in openssl.deb) because Debian's default openssl.cnf
raises the minimum required protocol / tls security level higher than
what are compiled into libssl1.1-udeb without a config file. As
otherwise the person who discovers that d-i can talk to an https
server, but in-target debian cannot will be rightfully confused.
Unless we decide that we don't care, as this is quite a niche corner
case.

-- 
Regards,

Dimitri.