Re: No Private Key found in '/etc/letsencrypt/live/www.mydomain.org/fullchain.pem.key

[Christoph Kukulies]

I'm normally not inept in compiling packages, using make, git, etc.  and I 
believe I got it right .

The thing is: haproxy 2.4 could cope with my config line, at least it didn't 
bail out on reading the config. Whether the option was in effect might have 
been another question.
Now, 2.8, complains about wrong syntax. I put it as a separate line in the 
global section at the beginning of the config file as someone - I believe it 
was Shawn - suggested.

Haproxy seems to be running now.
Is there a "testsuite" to test haproxy and its built in features from the 
outside?

--
Christoph

> Am 07.11.2023 um 14:38 schrieb Cyril Bonté :
> 
> Hi,
> 
> Le 07/11/2023 à 12:54, Christoph Kukulies a écrit :
>> (...) Now haproxy fails on my config (which the former version 2.4 I was 
>> running before, didn't)
>> This is the line in question:
>>  bind *:443 ssl crt /etc/haproxy/fullchain.pem crt ssl-skip-self-issued-ca
>> How do I fix this?
>> Put crt ssl-skip-self-issued-ca
>> in a separate line?
>> Where?
> 
> It seems you're not expert in compiling haproxy and you missed some options 
> (I guess SSL is not enabled), which may also be a problem for you future 
> upgrades.
> 
> Why not simply use available packages ?
> 
> See https://haproxy.debian.net/ and 
> https://www.haproxy.com/blog/how-to-install-haproxy-on-ubuntu
> 
> According to your previous information, it would result in:
> https://haproxy.debian.net/#distribution=Ubuntu=jammy=2.8
> 
> -- 
> Cyril Bonté
> 
> 



smime.p7s
Description: S/MIME cryptographic signature

Re: No Private Key found in '/etc/letsencrypt/live/www.mydomain.org/fullchain.pem.key

[Cyril Bonté]

Hi,

Le 07/11/2023 à 12:54, Christoph Kukulies a écrit :
(...) 
Now haproxy fails on my config (which the former version 2.4 I was 
running before, didn't)


This is the line in question:

  bind *:443 ssl crt /etc/haproxy/fullchain.pem crt ssl-skip-self-issued-ca

How do I fix this?

Put crt ssl-skip-self-issued-ca
in a separate line?

Where?


It seems you're not expert in compiling haproxy and you missed some 
options (I guess SSL is not enabled), which may also be a problem for 
you future upgrades.


Why not simply use available packages ?

See https://haproxy.debian.net/ and 
https://www.haproxy.com/blog/how-to-install-haproxy-on-ubuntu


According to your previous information, it would result in:
https://haproxy.debian.net/#distribution=Ubuntu=jammy=2.8

--
Cyril Bonté

Re: No Private Key found in '/etc/letsencrypt/live/www.mydomain.org/fullchain.pem.key

[Christoph Kukulies]

Finally got through the compilation of haproy-2.8, almost (had to apt  install 
libpcre2-dev )
to get around missing 
   from src/ssl_sock.c:47:
include/haproxy/regex-t.h:40:10: fatal error: pcre2.h: No such file or directory
   40 | #include 
> 

After having that fixed,compilation lastly stumbled across a missing 
fatal error: systemd/sd-daemon.h: No such file or directory

which I was able to repair by 

apt install libsystemd-dev

So I finally got it compiled.
Now haproxy fails on my config (which the former version 2.4 I was running 
before, didn't)

This is the line in question:

 bind *:443 ssl crt /etc/haproxy/fullchain.pem crt ssl-skip-self-issued-ca

How do I fix this? 

Put crt ssl-skip-self-issued-ca
in a separate line?

Where?

> Am 07.11.2023 um 12:27 schrieb Christoph Kukulies :
> 
> I found that a missing psutil module (python3) breaks the scripts. The # of 
> CPUs is gained from psutil query to
> give the make -j an argument. Missing argument should be intercepted.
> 
> I fixed this by installing psutil using pip3. (hope I didn't break my python 
> system environment)
> 
> Anyway, new-quic ran:
> 
> I 2023-11-07T12:17:36: Building quictls, may take a few minutes
> I 2023-11-07T12:17:36: Using 3 threads with make
> mv: cannot stat '/opt/quictls': No such file or directory
> I 2023-11-07T12:21:32: Installing quictls
> created directory `/opt/quictls'
> created directory `/opt/quictls/lib64'
> *** Installing runtime libraries
> install libcrypto.so.81.3 -> /opt/quictls/lib64/libcrypto.so.81.3
> install libssl.so.81.3 -> /opt/quictls/lib64/libssl.so.81.3
> *** Installing development files
> 
> ...
> 
> install apps/tsget.pl -> /opt/quictls/ssl/misc/tsget.pl
> link /opt/quictls/ssl/misc/tsget -> /opt/quictls/ssl/misc/tsget.pl
> install ./apps/openssl.cnf -> /opt/quictls/ssl/openssl.cnf.dist
> install ./apps/openssl.cnf -> /opt/quictls/ssl/openssl.cnf
> install ./apps/ct_log_list.cnf -> /opt/quictls/ssl/ct_log_list.cnf.dist
> install ./apps/ct_log_list.cnf -> /opt/quictls/ssl/ct_log_list.cnf
> rm -f libcrypto.so.81.3
> rm -f libcrypto.so
> rm -f libssl.so.81.3
> rm -f libssl.so
> root@mail:~/haproxy-scripts# 
> 
> 
>> Am 07.11.2023 um 11:53 schrieb Christoph Kukulies > >:
>> 
>> Thanks, Shawn.
>> I git-cloned all  links. I now have directories:
>> 
>> /root/haproxy-2.8
>> /root/haproxy-scripts
>> /root/openssl
>> 
>> 
>> I don't have a separate docker server available. Must do this in my 
>> production site (although it can tolerate short outages :)
>> 
>> I started the script new-haproxy (Ubuntu 22.04.3 LTS):
>> 
>> 
>> ...:
>> include/haproxy/openssl-compat.h:10:10: fatal error: openssl/bn.h: No such 
>> file or directory
>>10 | #include 
>>   |  ^~
>> compilation terminated.
>> compilation terminated.
>> 
>> (trimmed down the output a bit)
>> 
>> What is missing? zlib? 
>> 
>> --
>> Christoph
>> 
>> 
>>> Am 06.11.2023 um 00:38 schrieb Shawn Heisey >> >:
>>> 
>>> https://git.haproxy.org/git/haproxy-2.8.git 
>>> 
> 



smime.p7s
Description: S/MIME cryptographic signature

Re: No Private Key found in '/etc/letsencrypt/live/www.mydomain.org/fullchain.pem.key

[Christoph Kukulies]

I found that a missing psutil module (python3) breaks the scripts. The # of 
CPUs is gained from psutil query to
give the make -j an argument. Missing argument should be intercepted.

I fixed this by installing psutil using pip3. (hope I didn't break my python 
system environment)

Anyway, new-quic ran:

I 2023-11-07T12:17:36: Building quictls, may take a few minutes
I 2023-11-07T12:17:36: Using 3 threads with make
mv: cannot stat '/opt/quictls': No such file or directory
I 2023-11-07T12:21:32: Installing quictls
created directory `/opt/quictls'
created directory `/opt/quictls/lib64'
*** Installing runtime libraries
install libcrypto.so.81.3 -> /opt/quictls/lib64/libcrypto.so.81.3
install libssl.so.81.3 -> /opt/quictls/lib64/libssl.so.81.3
*** Installing development files

...

install apps/tsget.pl -> /opt/quictls/ssl/misc/tsget.pl
link /opt/quictls/ssl/misc/tsget -> /opt/quictls/ssl/misc/tsget.pl
install ./apps/openssl.cnf -> /opt/quictls/ssl/openssl.cnf.dist
install ./apps/openssl.cnf -> /opt/quictls/ssl/openssl.cnf
install ./apps/ct_log_list.cnf -> /opt/quictls/ssl/ct_log_list.cnf.dist
install ./apps/ct_log_list.cnf -> /opt/quictls/ssl/ct_log_list.cnf
rm -f libcrypto.so.81.3
rm -f libcrypto.so
rm -f libssl.so.81.3
rm -f libssl.so
root@mail:~/haproxy-scripts# 


> Am 07.11.2023 um 11:53 schrieb Christoph Kukulies :
> 
> Thanks, Shawn.
> I git-cloned all  links. I now have directories:
> 
> /root/haproxy-2.8
> /root/haproxy-scripts
> /root/openssl
> 
> 
> I don't have a separate docker server available. Must do this in my 
> production site (although it can tolerate short outages :)
> 
> I started the script new-haproxy (Ubuntu 22.04.3 LTS):
> 
> 
> ...:
> include/haproxy/openssl-compat.h:10:10: fatal error: openssl/bn.h: No such 
> file or directory
>10 | #include 
>   |  ^~
> compilation terminated.
> compilation terminated.
> 
> (trimmed down the output a bit)
> 
> What is missing? zlib? 
> 
> --
> Christoph
> 
> 
>> Am 06.11.2023 um 00:38 schrieb Shawn Heisey > >:
>> 
>> https://git.haproxy.org/git/haproxy-2.8.git 
>> 



smime.p7s
Description: S/MIME cryptographic signature

Re: No Private Key found in '/etc/letsencrypt/live/www.mydomain.org/fullchain.pem.key

[Christoph Kukulies]

Thanks, Shawn.
I git-cloned all  links. I now have directories:

/root/haproxy-2.8
/root/haproxy-scripts
/root/openssl


I don't have a separate docker server available. Must do this in my production 
site (although it can tolerate short outages :)

I started the script new-haproxy (Ubuntu 22.04.3 LTS):


root@mail:~/haproxy-scripts# ./new-haproxy haproxy-2.8
Traceback (most recent call last):
  File "", line 1, in 
ModuleNotFoundError: No module named 'psutil'
E 2023-11-07T11:49:19: GETTING CPU COUNT FAILED.
I 2023-11-07T11:49:19: - Beginning pull/build of haproxy
I 2023-11-07T11:49:19: Haproxy repo maintenance
Fetching haproxy from git
Fetching origin
Already up to date.
I 2023-11-07T11:49:20: Building haproxy
I 2023-11-07T11:49:20: Using  threads with make
  CC  src/ev_poll.o
  CC  src/ev_epoll.o
  CC  src/cpuset.o
  CC  src/ssl_sock.o
  CC  src/ssl_ckch.o
  CC  src/ssl_sample.o
  CC  src/ssl_crtlist.o
  CC  src/cfgparse-ssl.o
  CC  src/ssl_utils.o
  CC  src/jwt.o
  CC  src/ssl_ocsp.o
  CC  src/quic_conn.o
  CC  src/mux_quic.o
  CC  src/h3.o
 ..
  CC  src/quic_cc.o
  CC  src/quic_cc_nocc.o
  CC  src/namespace.o
  CC  src/linuxcap.o
  CC  src/mux_h2.o
  CC  src/mux_fcgi.o
  CC  src/mux_h1.o
  CC  src/tcpcheck.o
  CC  src/stream.o
  CC  src/stats.o
  CC  src/http_ana.o
In file included from include/haproxy/pattern-t.h:28,
 from include/haproxy/acl-t.h:25,
 from include/haproxy/acl.h:25,
 from src/ssl_sample.c:22:
include/haproxy/regex-t.h:40:10: fatal error: pcre2.h: No such file or directory
   40 | #include 
  |  ^
compilation terminated.
make: *** [Makefile:1009: src/ssl_sample.o] Error 1
make: *** Waiting for unfinished jobs
In file included from src/ssl_utils.c:18:
include/haproxy/openssl-compat.h:10:10: fatal error: openssl/bn.h: No such file 
or directory
   10 | #include 
  |  ^~
compilation terminated.
make: *** [Makefile:1009: src/ssl_utils.o] Error 1
In file included from src/cfgparse-ssl.c:38:
include/haproxy/openssl-compat.h:10:10: fatal error: openssl/bn.h: No such file 
or directory
   10 | #include 
  |  ^~
compilation terminated.


compilation terminated.
make: *** [Makefile:1009: src/quic_cc_cubic.o] Error 1
In file included from include/haproxy/proxy-t.h:35,
 from include/haproxy/obj_type.h:32,
 from include/haproxy/connection.h:34,
 from src/h3.c:24:
include/haproxy/compression-t.h:32:10: fatal error: zlib.h: No such file or 
directory
   32 | #include 
  |  ^~~~
In file included from include/haproxy/quic_conn.h:37,
 from src/quic_tp.c:7:
include/haproxy/openssl-compat.h:10:10: fatal error: openssl/bn.h: No such file 
or directory
   10 | #include 
  |  ^~
compilation terminated.
compilation terminated.

(trimmed down the output a bit)

What is missing? zlib? 

--
Christoph


> Am 06.11.2023 um 00:38 schrieb Shawn Heisey :
> 
> https://git.haproxy.org/git/haproxy-2.8.git 
> 


smime.p7s
Description: S/MIME cryptographic signature

Re: No Private Key found in '/etc/letsencrypt/live/www.mydomain.org/fullchain.pem.key

[Shawn Heisey]

On 11/5/2023 02:48, Christoph Kukulies wrote:

I git cloned haproxy and compiled it :

root@mail:~/haproxy# ./haproxy --version
HAProxy version 2.9-dev8-ce7501-38 2023/11/04 - https://haproxy.org/ 


Status: development branch - not safe for use in production.
Known bugs: https://github.com/haproxy/haproxy/issues?q=is:issue+is:open 

Running on: Linux 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 
UTC 2023 x86_64
Usage : haproxy [-f ]* [ -vdVD ] [ -n  ] [ -N 
 ]


Probably this is not what I want? Better 2.8 stable?
I compiled with

make TARGET=linux-glibc


Many projects have a single git repo for all versions and use branches 
to separate them.  Haproxy doesn't.  2.8 is developed in a completely 
separate git repository from the one that you cloned.  This is the repo 
that you want:


https://git.haproxy.org/git/haproxy-2.8.git

My scripts just make things easier.  They will compile/install haproxy 
2.8 and the latest 3.1.x version of quictls/openssl (currently 3.1.4) 
with only a few commands.  The repo does not contain binaries ... all 
scripts can be examined to verify that nothing shady is happening. 
Today I pushed up some fixes.


The quictls repo is a fork of openssl, which has been patched to include 
QUIC functions that haproxy can use to provide QUIC/HTTP3:

https://github.com/quictls/openssl

Thanks,
Shawn

Re: No Private Key found in '/etc/letsencrypt/live/www.mydomain.org/fullchain.pem.key

[Christoph Kukulies]

I git cloned haproxy and compiled it :

root@mail:~/haproxy# ./haproxy --version
HAProxy version 2.9-dev8-ce7501-38 2023/11/04 - https://haproxy.org/
Status: development branch - not safe for use in production.
Known bugs: https://github.com/haproxy/haproxy/issues?q=is:issue+is:open
Running on: Linux 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 
x86_64
Usage : haproxy [-f ]* [ -vdVD ] [ -n  ] [ -N 
 ]

Probably this is not what I want? Better 2.8 stable? 
I compiled with

make TARGET=linux-glibc

--
Christoph

> Am 04.11.2023 um 08:42 schrieb Christoph Kukulies :
> 
> I was informed off list, that putting the ssl-skip-self-issued-ca inline like 
> I 
> did, would make haproxy fila on the config.
> 
> It doesn't do so on me here with my config.
> As mentioned, my haproxy is 2.4.22. This is what I got with ubuntu apt 
> install.
> 
> How does one install haproxy directly under Ubuntu, also to be more up to 
> date?
> 
> --
> Christoph
> 
> 
>> Am 03.11.2023 um 09:49 schrieb Christoph Kukulies > >:
>> 
>> Thanks, Shawn,
>> 
>> I always have my problems with the open form of the configuration file 
>> syntax (lua ?).
>> The docs say it is a keyword under "crt" which in turn belongs to the "bind" 
>> options.
>> 
>> Would it be correct to place it that way?:
>> 
>> frontend http-in
>> bind *:80
>> bind *:443 ssl crt /etc/haproxy/fullchain.pem crt ssl-skip-self-issued-ca
>> 
>> 
>>> Am 03.11.2023 um 03:50 schrieb Shawn Heisey >> >:
>>> 
>>> On 11/2/2023 02:35, Christoph Kukulies wrote:
 In /etc/letsencrypt/live/www.mydomain.org  I 
 have:
 lrwxrwxrwx 1 root root  41 Oct 23 17:22 *cert.pem*-> 
 ../../archive/www.mydomain.org/cert12.pem 
  >
 lrwxrwxrwx 1 root root  42 Oct 23 17:22 *chain.pem*-> 
 ../../archive/www.mydomain.org/chain12.pem 
  >
 lrwxrwxrwx 1 root root  46 Oct 23 17:22 *fullchain.pem*-> 
 ../../archive/www.mydomain.org/fullchain12.pem 
  
 >
 lrwxrwxrwx 1 root root  13 Nov  1 12:12 *fullchain.pem.key*-> fullchain.pem
 lrwxrwxrwx 1 root root  44 Oct 23 17:22 *privkey.pem*-> 
 ../../archive/www.mydomain.org/privkey12.pem 
  
 >
 lrwxrwxrwx 1 root root  11 Nov  1 12:11 *privkey.pem.key*-> privkey.pem
 -rw-r--r-- 1 root root 692 Nov 13  2021 README
 But note, that the file ending on .key are put there on an expermental 
 basis, because I read somewhere in the haproxy docs that one could a file 
 with extension .key
 there and haproxy then adds interprets that as the private key. Location 
 for this hint escaped me for the moment.
>>> 
>>> The link named 'fullchain.pem.key' is not pointing at a key.  It is 
>>> pointing at the fullchain, which as already mentioned, does NOT contain the 
>>> private key.
>>> 
>>> If you change that symlink to point at privkey.pem instead of 
>>> fullchain.pem, haproxy might start working.  You do not need the 
>>> privkey.pem.key symlink.
>>> 
>>> If you're going to use the fullchain file in haproxy, then you should also 
>>> use the ssl-skip-self-issued-ca config that William mentioned so the root 
>>> cert is not sent to browsers.
>>> 
>>> Thanks,
>>> Shawn
>>> 
>> 
>> --
>> Christoph
>> 
>> 
> 



smime.p7s
Description: S/MIME cryptographic signature

Re: No Private Key found in '/etc/letsencrypt/live/www.mydomain.org/fullchain.pem.key

[Shawn Heisey]

On 11/4/2023 01:42, Christoph Kukulies wrote:
How does one install haproxy directly under Ubuntu, also to be more up 
to date?


I created this set of scripts that will automate the build and install 
of the latest haproxy 2.8 version with support for HTTP/3.  It builds 
directly from the 2.8 dev repo, so what it installs may be even newer 
than the most recent 2.8.x release:


https://github.com/elyograg/haproxy-scripts

The prep-source script installs a whole bunch of packages -- everything 
that is needed to build quictls and haproxy.  It also modifies 
/etc/apt/sources.list to uncomment the source repos.  If you have not 
touched your /etc/apt/sources.list file, it should work perfectly and 
not break your APT setup.


I built this to work on Ubuntu.  It has been well-tested on Ubuntu, but 
it should also work on RHEL and its derivatives.


Although I do include a sample haproxy config, it is not really suitable 
as-is for production.  It is a config used by the gitlab CI/CD that I 
built for the project.


Thanks,
Shawn

Re: No Private Key found in '/etc/letsencrypt/live/www.mydomain.org/fullchain.pem.key

[Christoph Kukulies]

I was informed off list, that putting the ssl-skip-self-issued-ca inline like I 
did, would make haproxy fila on the config.

It doesn't do so on me here with my config.
As mentioned, my haproxy is 2.4.22. This is what I got with ubuntu apt install.

How does one install haproxy directly under Ubuntu, also to be more up to date?

--
Christoph


> Am 03.11.2023 um 09:49 schrieb Christoph Kukulies :
> 
> Thanks, Shawn,
> 
> I always have my problems with the open form of the configuration file syntax 
> (lua ?).
> The docs say it is a keyword under "crt" which in turn belongs to the "bind" 
> options.
> 
> Would it be correct to place it that way?:
> 
> frontend http-in
> bind *:80
> bind *:443 ssl crt /etc/haproxy/fullchain.pem crt ssl-skip-self-issued-ca
> 
> 
>> Am 03.11.2023 um 03:50 schrieb Shawn Heisey > >:
>> 
>> On 11/2/2023 02:35, Christoph Kukulies wrote:
>>> In /etc/letsencrypt/live/www.mydomain.org  I have:
>>> lrwxrwxrwx 1 root root  41 Oct 23 17:22 *cert.pem*-> 
>>> ../../archive/www.mydomain.org/cert12.pem 
>>>  >> >
>>> lrwxrwxrwx 1 root root  42 Oct 23 17:22 *chain.pem*-> 
>>> ../../archive/www.mydomain.org/chain12.pem 
>>>  >> >
>>> lrwxrwxrwx 1 root root  46 Oct 23 17:22 *fullchain.pem*-> 
>>> ../../archive/www.mydomain.org/fullchain12.pem 
>>>  
>>> >> >
>>> lrwxrwxrwx 1 root root  13 Nov  1 12:12 *fullchain.pem.key*-> fullchain.pem
>>> lrwxrwxrwx 1 root root  44 Oct 23 17:22 *privkey.pem*-> 
>>> ../../archive/www.mydomain.org/privkey12.pem 
>>>  
>>> >> >
>>> lrwxrwxrwx 1 root root  11 Nov  1 12:11 *privkey.pem.key*-> privkey.pem
>>> -rw-r--r-- 1 root root 692 Nov 13  2021 README
>>> But note, that the file ending on .key are put there on an expermental 
>>> basis, because I read somewhere in the haproxy docs that one could a file 
>>> with extension .key
>>> there and haproxy then adds interprets that as the private key. Location 
>>> for this hint escaped me for the moment.
>> 
>> The link named 'fullchain.pem.key' is not pointing at a key.  It is pointing 
>> at the fullchain, which as already mentioned, does NOT contain the private 
>> key.
>> 
>> If you change that symlink to point at privkey.pem instead of fullchain.pem, 
>> haproxy might start working.  You do not need the privkey.pem.key symlink.
>> 
>> If you're going to use the fullchain file in haproxy, then you should also 
>> use the ssl-skip-self-issued-ca config that William mentioned so the root 
>> cert is not sent to browsers.
>> 
>> Thanks,
>> Shawn
>> 
> 
> --
> Christoph
> 
> 



smime.p7s
Description: S/MIME cryptographic signature

Re: No Private Key found in '/etc/letsencrypt/live/www.mydomain.org/fullchain.pem.key

[Christoph Kukulies]

Thanks, Shawn,

I always have my problems with the open form of the configuration file syntax 
(lua ?).
The docs say it is a keyword under "crt" which in turn belongs to the "bind" 
options.

Would it be correct to place it that way?:

frontend http-in
bind *:80
bind *:443 ssl crt /etc/haproxy/fullchain.pem crt ssl-skip-self-issued-ca


> Am 03.11.2023 um 03:50 schrieb Shawn Heisey :
> 
> On 11/2/2023 02:35, Christoph Kukulies wrote:
>> In /etc/letsencrypt/live/www.mydomain.org I have:
>> lrwxrwxrwx 1 root root  41 Oct 23 17:22 *cert.pem*-> 
>> ../../archive/www.mydomain.org/cert12.pem 
>> 
>> lrwxrwxrwx 1 root root  42 Oct 23 17:22 *chain.pem*-> 
>> ../../archive/www.mydomain.org/chain12.pem 
>> 
>> lrwxrwxrwx 1 root root  46 Oct 23 17:22 *fullchain.pem*-> 
>> ../../archive/www.mydomain.org/fullchain12.pem 
>> 
>> lrwxrwxrwx 1 root root  13 Nov  1 12:12 *fullchain.pem.key*-> fullchain.pem
>> lrwxrwxrwx 1 root root  44 Oct 23 17:22 *privkey.pem*-> 
>> ../../archive/www.mydomain.org/privkey12.pem 
>> 
>> lrwxrwxrwx 1 root root  11 Nov  1 12:11 *privkey.pem.key*-> privkey.pem
>> -rw-r--r-- 1 root root 692 Nov 13  2021 README
>> But note, that the file ending on .key are put there on an expermental 
>> basis, because I read somewhere in the haproxy docs that one could a file 
>> with extension .key
>> there and haproxy then adds interprets that as the private key. Location for 
>> this hint escaped me for the moment.
> 
> The link named 'fullchain.pem.key' is not pointing at a key.  It is pointing 
> at the fullchain, which as already mentioned, does NOT contain the private 
> key.
> 
> If you change that symlink to point at privkey.pem instead of fullchain.pem, 
> haproxy might start working.  You do not need the privkey.pem.key symlink.
> 
> If you're going to use the fullchain file in haproxy, then you should also 
> use the ssl-skip-self-issued-ca config that William mentioned so the root 
> cert is not sent to browsers.
> 
> Thanks,
> Shawn
> 

--
Christoph




smime.p7s
Description: S/MIME cryptographic signature

Re: No Private Key found in '/etc/letsencrypt/live/www.mydomain.org/fullchain.pem.key

[Shawn Heisey]

On 11/2/2023 02:35, Christoph Kukulies wrote:

In /etc/letsencrypt/live/www.mydomain.org I have:

lrwxrwxrwx 1 root root  41 Oct 23 17:22 *cert.pem*-> 
../../archive/www.mydomain.org/cert12.pem 

lrwxrwxrwx 1 root root  42 Oct 23 17:22 *chain.pem*-> 
../../archive/www.mydomain.org/chain12.pem 

lrwxrwxrwx 1 root root  46 Oct 23 17:22 *fullchain.pem*-> 
../../archive/www.mydomain.org/fullchain12.pem 


lrwxrwxrwx 1 root root  13 Nov  1 12:12 *fullchain.pem.key*-> fullchain.pem
lrwxrwxrwx 1 root root  44 Oct 23 17:22 *privkey.pem*-> 
../../archive/www.mydomain.org/privkey12.pem 


lrwxrwxrwx 1 root root  11 Nov  1 12:11 *privkey.pem.key*-> privkey.pem
-rw-r--r-- 1 root root 692 Nov 13  2021 README

But note, that the file ending on .key are put there on an expermental 
basis, because I read somewhere in the haproxy docs that one could a 
file with extension .key
there and haproxy then adds interprets that as the private key. Location 
for this hint escaped me for the moment.


The link named 'fullchain.pem.key' is not pointing at a key.  It is 
pointing at the fullchain, which as already mentioned, does NOT contain 
the private key.


If you change that symlink to point at privkey.pem instead of 
fullchain.pem, haproxy might start working.  You do not need the 
privkey.pem.key symlink.


If you're going to use the fullchain file in haproxy, then you should 
also use the ssl-skip-self-issued-ca config that William mentioned so 
the root cert is not sent to browsers.


Thanks,
Shawn

Re: No Private Key found in '/etc/letsencrypt/live/www.mydomain.org/fullchain.pem.key

[Christoph Kukulies]

William, Shawn, 

excuses for responding with delay. I've been offline for the past 24 hours.
Here more infomation:

root@mail:/etc/haproxy# /usr/sbin/haproxy --version
HAProxy version 2.4.22-0ubuntu0.22.04.2 2023/08/14 - https://haproxy.org/
Status: long-term supported branch - will stop receiving fixes around Q2 2026.
Known bugs: http://www.haproxy.org/bugs/bugs-2.4.22.html
Running on: Linux 5.15.0-87-generic #97-Ubuntu SMP Mon Oct 2 21:09:21 UTC 2023 
x86_64


Here is my haproxy.cfg.

global
log /dev/loglocal0
log /dev/loglocal1 notice
chroot /var/lib/haproxy
pidfile /var/run/haproxy
stats socket /run/haproxy/admin.sock mode 660 level admin
stats timeout 30s
maxconn 4000
user haproxy
group haproxy
daemon

# Default SSL material locations
ca-base /etc/ssl/apache2/certs
crt-base /etc/ssl/apache2/private

# Default ciphers to use on SSL-enabled listening sockets.
# For more information, see ciphers(1SSL). This list is from:
#  https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/
ssl-default-bind-ciphers 
ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS
ssl-dh-param-file /etc/haproxy/dhparams.pem
ssl-default-bind-options no-sslv3

defaults
log global
modehttp
option  httplog
option  dontlognull
timeout connect 5000
timeout client  5
timeout server  5
errorfile 400 /etc/haproxy/errors/400.http
errorfile 403 /etc/haproxy/errors/403.http
errorfile 408 /etc/haproxy/errors/408.http
errorfile 500 /etc/haproxy/errors/500.http
errorfile 502 /etc/haproxy/errors/502.http
errorfile 503 /etc/haproxy/errors/503.http
errorfile 504 /etc/haproxy/errors/504.http
log global
mode http
compression algo gzip
compression type text/html text/css text/plain text/vcard 
text/vnd.rim.location.xloc text/vtt text/x-component text/x-cross-domain-policy 
application/atom+xml application/javascript application/x-javascript 
application/json application/ld+json application/manifest+json 
application/rss+xml application/vnd.geo+json application/vnd.ms-fontobject 
application/x-font-ttf application/x-web-app-manifest+json 
application/xhtml+xml application/xml font/opentype image/bmp image/svg+xml 
image/x-icon text/cache-manifest
balance roundrobin
option dontlog-normal
option dontlognull
option httpclose
option forwardfor

frontend http-in
bind *:80
bind *:443 ssl crt /etc/letsencrypt/live/www.mydomain.org/fullchain.pem

# Redirect if HTTPS is *not* used
redirect scheme https code 301 if !{ ssl_fc }

acl letsencrypt-acl path_beg /.well-known/acme-challenge/

use_backend letsencrypt-backend if letsencrypt-acl
default_backend website

log /dev/log local2 debug

acl is_static   path_beg /export/ /opencms/ /resources/ /javadoc/ 
/VAADIN/ /workplace /opencms-login/
acl is_website  hdr_beg(host) -i www.mydomain.org
use_backend website-static if is_website is_static
use_backend website if is_website

backend letsencrypt-backend
server letsencrypt 127.0.0.1:

backend website-static
server www.mydomain.org 127.0.0.1:8080

backend website
reqirep ^([^\ :]*)\ /(.*) \1\ /opencms/\2
server www.mydomain.org 127.0.0.1:8080

> Am 01.11.2023 um 23:14 schrieb William Lallemand :
> 
In /etc/letsencrypt/live/www.mydomain.org  I have:

lrwxrwxrwx 1 root root  41 Oct 23 17:22 cert.pem -> 
../../archive/www.mydomain.org/cert12.pem
lrwxrwxrwx 1 root root  42 Oct 23 17:22 chain.pem -> 
../../archive/www.mydomain.org/chain12.pem
lrwxrwxrwx 1 root root  46 Oct 23 17:22 fullchain.pem -> 
../../archive/www.mydomain.org/fullchain12.pem
lrwxrwxrwx 1 root root  13 Nov  1 12:12 fullchain.pem.key -> fullchain.pem
lrwxrwxrwx 1 root root  44 Oct 23 17:22 privkey.pem -> 
../../archive/www.mydomain.org/privkey12.pem
lrwxrwxrwx 1 root root  11 Nov  1 12:11 privkey.pem.key -> privkey.pem
-rw-r--r-- 1 root root 692 Nov 13  2021 README

But note, that the file ending on .key are put there on an expermental basis, 
because I read somewhere in the haproxy docs that one could a file with 
extension .key
there and haproxy then adds interprets that as the private key. Location for 
this hint escaped me for the moment.

--
Christoph



> On Wed, Nov 01, 2023 at 03:48:56PM -0600, Shawn Heisey wrote:
>> The LE fullchain file does not contain the key.  It contains 3 
>> certificates. ... the server cert, the issuing cert, and the root cert 
>> ... which is not what you want.  For letsencrypt, the file that you give 
>> to haproxy must contain the server cert, the issuing cert, and the 
>> private key.
> 
> Actually you can separate the key from the chain with haproxy, but
> without 

Re: No Private Key found in '/etc/letsencrypt/live/www.mydomain.org/fullchain.pem.key

[William Lallemand]

On Wed, Nov 01, 2023 at 03:48:56PM -0600, Shawn Heisey wrote:
> The LE fullchain file does not contain the key.  It contains 3 
> certificates. ... the server cert, the issuing cert, and the root cert 
> ... which is not what you want.  For letsencrypt, the file that you give 
> to haproxy must contain the server cert, the issuing cert, and the 
> private key.

Actually you can separate the key from the chain with haproxy, but
without the configuration it's difficult to know what it's trying to
load.

http://docs.haproxy.org/2.8/configuration.html#ssl-load-extra-files

With the 'key' option it's able to load a 'file.pem.key' if you
specified 'crt file.pem' in your configuration


> You do not want to include the root certificate.  It will be ignored
> by the browser even if it is included, but it will probably slow down
> TLS negotiation by a small amount.  The presence of the root
> certificate in the TLS handshake should not actually break anything in
> most cases, but it could result in a lower score on the Qualys Labs
> SSL test.
>

You can also ask haproxy to ignore the root CA in the configuration:

http://docs.haproxy.org/2.8/configuration.html#ssl-skip-self-issued-ca


> When my renewal script finishes, I have a file containing four things: 
> The server cert, the issuing cert, the private key, and a unique 4096 
> bit DHPARAM.  This combination is ideal for haproxy.
> 
> The version of certbot that I am using generates 256-bit ECDSA keys by 
> default.  You might be thinking that a 256 bit ECDSA key cannot be as 
> secure as a 2048 bit RSA key, but that is incorrect:
> 
> https://www.baeldung.com/cs/encryption-asymmetric-algorithms#3-key-length
> 
> Some of the equipment I use will not work with ECDSA keys, so I have a 
> second cert with a subset of names that I build using 4096 bit RSA.
> 

There is a lot of possibility to configure this, we are trying to move
forward to a configuration where we can specifiy files separately so
there would be nothing to do in the future, but unfortunately there is
still development to do.

-- 
William Lallemand

Re: No Private Key found in '/etc/letsencrypt/live/www.mydomain.org/fullchain.pem.key

[Shawn Heisey]

On 11/1/23 05:20, Christoph Kukulies wrote:
'bind *:443' : No Private Key found in 
'/etc/letsencrypt/live/www.mydomain.org/fullchain.pem.key' 
.


I have the following in my
/etc/letsencrypt/live/www.mydomain.org :

lrwxrwxrwx 1 root root  41 Oct 23 17:22 cert.pem -> 
../../archive/www.mydomain.org/cert12.pem
lrwxrwxrwx 1 root root  42 Oct 23 17:22 chain.pem -> 
../../archive/www.mydomain.org/chain12.pem
lrwxrwxrwx 1 root root  46 Oct 23 17:22 fullchain.pem -> 
../../archive/www.mydomain.org/fullchain12.pem

lrwxrwxrwx 1 root root  13 Nov  1 12:12 fullchain.pem.key -> fullchain.pem
lrwxrwxrwx 1 root root  44 Oct 23 17:22 privkey.pem -> 
../../archive/www.mydomain.org/privkey12.pem

lrwxrwxrwx 1 root root  11 Nov  1 12:11 privkey.pem.key -> privkey.pem
-rw-r--r-- 1 root root 692 Nov 13  2021 README


This is what I have:

root@smeagol:/etc/letsencrypt/archive/elyograg.org-0022# ls -al 
/etc/letsencrypt/live/elyograg.org-0022

total 12
drwxr-xr-x  2 root root 4096 Nov  1 00:00 .
drwx-- 53 root root 4096 Nov  1 00:02 ..
lrwxrwxrwx  1 root root   41 Nov  1 00:00 cert.pem -> 
../../archive/elyograg.org-0022/cert1.pem
lrwxrwxrwx  1 root root   42 Nov  1 00:00 chain.pem -> 
../../archive/elyograg.org-0022/chain1.pem
lrwxrwxrwx  1 root root   46 Nov  1 00:00 fullchain.pem -> 
../../archive/elyograg.org-0022/fullchain1.pem
lrwxrwxrwx  1 root root   44 Nov  1 00:00 privkey.pem -> 
../../archive/elyograg.org-0022/privkey1.pem

-rw-r--r--  1 root root  692 Nov  1 00:00 README
root@smeagol:/etc/letsencrypt/archive/elyograg.org-0022# ls -al
total 28
drwxr-xr-x  2 root root 4096 Nov  1 00:00 .
drwx-- 53 root root 4096 Nov  1 00:02 ..
-rw-r--r--  1 root root 2329 Nov  1 00:00 cert1.pem
-rw-r--r--  1 root root 3749 Nov  1 00:00 chain1.pem
-rw-r--r--  1 root root 6078 Nov  1 00:00 fullchain1.pem
-rw---  1 root root  241 Nov  1 00:00 privkey1.pem

The LE fullchain file does not contain the key.  It contains 3 
certificates. ... the server cert, the issuing cert, and the root cert 
... which is not what you want.  For letsencrypt, the file that you give 
to haproxy must contain the server cert, the issuing cert, and the 
private key.  You do not want to include the root certificate.  It will 
be ignored by the browser even if it is included, but it will probably 
slow down TLS negotiation by a small amount.  The presence of the root 
certificate in the TLS handshake should not actually break anything in 
most cases, but it could result in a lower score on the Qualys Labs SSL 
test.


When my renewal script finishes, I have a file containing four things: 
The server cert, the issuing cert, the private key, and a unique 4096 
bit DHPARAM.  This combination is ideal for haproxy.


The version of certbot that I am using generates 256-bit ECDSA keys by 
default.  You might be thinking that a 256 bit ECDSA key cannot be as 
secure as a 2048 bit RSA key, but that is incorrect:


https://www.baeldung.com/cs/encryption-asymmetric-algorithms#3-key-length

Some of the equipment I use will not work with ECDSA keys, so I have a 
second cert with a subset of names that I build using 4096 bit RSA.


Thanks,
Shawn

Re: No Private Key found in '/etc/letsencrypt/live/www.mydomain.org/fullchain.pem.key

[William Lallemand]

Hello,

Please share at least your haproxy configuration and version and what
you are trying to do, your mail is really unclear.

On Wed, Nov 01, 2023 at 12:20:35PM +0100, Christoph Kukulies wrote:
> 'bind *:443' : No Private Key found in 
> '/etc/letsencrypt/live/www.mydomain.org/fullchain.pem.key'.
> 
> I have the following in my
> /etc/letsencrypt/live/www.mydomain.org:
> 
> lrwxrwxrwx 1 root root  41 Oct 23 17:22 cert.pem -> 
> ../../archive/www.mydomain.org/cert12.pem
> lrwxrwxrwx 1 root root  42 Oct 23 17:22 chain.pem -> 
> ../../archive/www.mydomain.org/chain12.pem
> lrwxrwxrwx 1 root root  46 Oct 23 17:22 fullchain.pem -> 
> ../../archive/www.mydomain.org/fullchain12.pem
> lrwxrwxrwx 1 root root  13 Nov  1 12:12 fullchain.pem.key -> fullchain.pem
> lrwxrwxrwx 1 root root  44 Oct 23 17:22 privkey.pem -> 
> ../../archive/www.mydomain.org/privkey12.pem
> lrwxrwxrwx 1 root root  11 Nov  1 12:11 privkey.pem.key -> privkey.pem
> -rw-r--r-- 1 root root 692 Nov 13  2021 README
> 
> 
> Before I had no .key files in there.
> 
> Since I added the symbolic links to the files with .key extension I got an 
> additional line in the log:
> 
> Nov  1 12:13:40 mail haproxy[103552]:
> '/etc/letsencrypt/live/www.mydomain.org/fullchain.pem.key' is present but 
> cannot be read or parsed'.
> 
> I'm a bit clueless now. Help appreciated.
> 
> --
> Christoph
> 



-- 
William Lallemand